Sep 10 14:40:31.373270 ip-10-0-142-253 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 14:40:31.373280 ip-10-0-142-253 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 14:40:31.373290 ip-10-0-142-253 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 14:40:31.373632 ip-10-0-142-253 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 14:40:41.448852 ip-10-0-142-253 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 14:40:41.448872 ip-10-0-142-253 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e014573e81ae4b24ade289dcd4f945f4 --
Sep 10 14:42:35.402523 ip-10-0-142-253 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 14:42:35.782109 ip-10-0-142-253 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:42:35.782109 ip-10-0-142-253 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 14:42:35.782109 ip-10-0-142-253 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:42:35.782109 ip-10-0-142-253 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 14:42:35.782109 ip-10-0-142-253 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:42:35.782826 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.782169    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 14:42:35.788052 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788034    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:42:35.788052 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788052    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788058    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788063    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788066    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788069    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788073    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788075    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788078    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788081    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788083    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788086    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788089    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788091    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788094    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788096    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788102    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788105    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788108    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788111    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788114    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:42:35.788127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788116    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788119    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788122    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788125    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788128    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788131    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788134    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788137    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788140    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788143    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788146    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788148    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788151    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788154    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788157    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788160    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788162    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788165    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788167    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788170    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:42:35.788612 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788172    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788175    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788178    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788180    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788183    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788186    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788191    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788193    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788196    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788199    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788203    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788206    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788210    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788212    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788231    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788234    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788237    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788240    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788243    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:42:35.789087 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788246    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788249    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788252    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788255    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788258    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788261    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788264    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788266    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788269    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788272    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788274    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788277    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788283    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788286    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788289    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788292    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788295    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788298    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788301    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:42:35.789589 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788304    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788307    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788311    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788313    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788316    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788318    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788321    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788324    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788327    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788330    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788333    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788335    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788338    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788340    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788343    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788346    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788348    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788351    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788354    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.788357    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:42:35.790041 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789068    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789075    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789078    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789082    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789085    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789088    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789091    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789094    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789097    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789099    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789102    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789105    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789108    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789110    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789113    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789116    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789119    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789122    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789124    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789128    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:42:35.790551 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789131    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789134    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789137    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789139    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789143    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789145    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789148    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789151    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789155    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789159    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789162    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789164    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789167    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789170    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789173    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789177    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789181    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789183    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789186    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:42:35.791040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789189    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789191    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789194    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789196    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789198    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789201    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789204    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789206    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789209    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789212    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789230    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789234    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789237    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789240    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789242    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789245    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789248    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789250    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789253    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789255    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:42:35.791521 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789258    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789261    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789263    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789266    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789269    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789271    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789274    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789276    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789279    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789281    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789284    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789286    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789290    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789292    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789295    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789298    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789300    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789304    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789308    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:42:35.792006 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789311    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789313    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789316    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789320    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789322    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789325    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789327    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789330    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789332    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789335    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789338    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789340    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789343    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789345    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789348    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789350    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789353    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789355    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789358    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789360    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:42:35.792495 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789363    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789767    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789774    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789778    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789781    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789784    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789787    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789790    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789793    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789796    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789799    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789802    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789805    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789807    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789810    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789813    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789815    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789818    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789821    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:42:35.792967 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789823    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789826    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789828    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789831    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789833    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789836    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789838    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789841    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789843    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789846    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789848    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789853    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789855    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789858    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789860    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789863    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789865    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789868    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789870    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:42:35.793427 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789873    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789875    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789878    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789881    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789883    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789886    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789889    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789891    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789894    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789900    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789903    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789905    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789908    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789910    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789913    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789915    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789918    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789921    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789925    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789927    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:42:35.793892 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789930    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789932    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789935    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789938    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789942    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789945    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789947    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789950    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789952    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789956    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789959    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789962    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789965    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789968    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789970    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789973    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789976    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789978    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789981    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789984    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:42:35.794429 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789987    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789989    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789993    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789995    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.789998    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790001    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790004    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790006    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790009    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790011    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790015    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790017    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790020    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790022    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790024    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790027    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790031    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790033    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790036    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790039    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:42:35.794921 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790042    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790045    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790120    2558 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790159    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790166    2558 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790170    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790175    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790178    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790182    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790186    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790189    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790193    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790196    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790199    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790203    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790206    2558 flags.go:64] FLAG: --cgroup-root=""
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790209    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790212    2558 flags.go:64] FLAG: --client-ca-file=""
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790230    2558 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790233    2558 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790237    2558 flags.go:64] FLAG: --cluster-domain=""
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790240    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790245    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790248    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 14:42:35.795403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790251    2558 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790255    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790258    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790261    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790265    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790268    2558 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790272    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790275    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790278    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790281    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790285    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790288    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790291    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790294    2558 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790297    2558 flags.go:64] FLAG: --enable-server="true"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790299    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790304    2558 flags.go:64] FLAG: --event-burst="100"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790307    2558 flags.go:64] FLAG: --event-qps="50"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790310    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790313    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790316    2558 flags.go:64] FLAG: --eviction-hard=""
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790319    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790322    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790325    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790329    2558 flags.go:64] FLAG: --eviction-soft=""
Sep 10 14:42:35.796028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790331    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790334    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790337    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790340    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790342    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790346    2558 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790349    2558 flags.go:64] FLAG: --feature-gates=""
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790353    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790356    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790359    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790362    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790366    2558 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790369    2558 flags.go:64] FLAG: --help="false"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790372    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-142-253.ec2.internal"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790375    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790377    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790380    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790383    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790387    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790390    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790392    2558 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790395    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790398    2558 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 14:42:35.796643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790401    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790404    2558 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790406    2558 flags.go:64] FLAG: --kube-reserved=""
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790409    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790412    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790415    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790417    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790420    2558 flags.go:64] FLAG: --lock-file=""
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790422    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790426    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790429    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790434    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790437    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790439    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790443    2558 flags.go:64] FLAG: --logging-format="text"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790446    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790449    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790452    2558 flags.go:64] FLAG: --manifest-url=""
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790454    2558 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790459    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790463    2558 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790467    2558 flags.go:64] FLAG: --max-pods="110"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790470    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790473    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790476    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 14:42:35.797234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790479    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790482    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790485    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790487    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790494    2558 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790497    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790500    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790503    2558 flags.go:64] FLAG: --pod-cidr=""
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790506    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790511    2558 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790514    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790517    2558 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790519    2558 flags.go:64] FLAG: --port="10250"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790522    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790525    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0713e79c939189ce3"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790528    2558 flags.go:64] FLAG: --qos-reserved=""
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790531    2558 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790535    2558 flags.go:64] FLAG: --register-node="true"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790537    2558 flags.go:64] FLAG: --register-with-taints=""
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790541    2558 flags.go:64] FLAG: --registry-burst="10"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790543    2558 flags.go:64] FLAG: --registry-qps="5"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790547    2558 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790550    2558 flags.go:64] FLAG: --reserved-memory=""
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790554    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790557    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 14:42:35.797886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790560    2558 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790563    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790566    2558 flags.go:64] FLAG: --runonce="false"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790570    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790573    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790575    2558 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790578    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790581    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790584    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790587    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790590    2558 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790593    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790596    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790598    2558 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790601    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790604    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790607    2558 flags.go:64] FLAG: --system-cgroups=""
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790609    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790614    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790617    2558 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790620    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790624    2558 flags.go:64] FLAG: --tls-min-version=""
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790626    2558 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790629    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790632    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 14:42:35.798599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790636    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790639    2558 flags.go:64] FLAG: --v="2"
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790643    2558 flags.go:64] FLAG: --version="false"
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790648    2558 flags.go:64] FLAG: --vmodule=""
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790652    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.790655    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790772    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790777    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790780    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790783    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790787    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790790    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790793    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790795    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790798    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790800    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790803    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790806    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790808    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790811    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:42:35.799309 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790814    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790818    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790822    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790824    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790829    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790832    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790835    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790838    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790841    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790844    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790847    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790850    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790852    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790856    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790858    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790862    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790865    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790868    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790871    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:42:35.799792 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790874    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790876    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790879    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790881    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790884    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790887    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790889    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790892    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790894    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790896    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790899    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790901    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790904    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790906    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790909    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790911    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790914    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790916    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790918    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790921    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:42:35.800267 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790923    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790926    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790928    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790931    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790933    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790936    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790938    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790941    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790946    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790948    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790951    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790953    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790956    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790958    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790961    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790963    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790966    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790968    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790971    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790973    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:42:35.800764 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790976    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790978    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790981    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790983    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790986    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790988    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790990    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790993    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790995    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.790998    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791000    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791002    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791005    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791007    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791009    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791012    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791015    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791018    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791023    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791026    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:42:35.801264 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791032    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791035    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791038    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791040    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791043    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.791045    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.791833    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.799596    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.799614    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.799628    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.799634    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799691    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799696    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799699    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799702    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:42:35.801746 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799705    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799709    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799711    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799714    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799717    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799719    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799722    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799725    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799728    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799730    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799733    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799735    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799738    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799740    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799743    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799746    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799748    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799751    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799754    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799757    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:42:35.802129 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799759    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799762    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799765    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799767    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799770    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799773    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799775    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799779    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799781    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799784    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799787    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799789    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799792    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799794    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799797    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799800    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799803    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799806    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799809    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799811    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:42:35.802631 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799814    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799817    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799819    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799822    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799825    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799827    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799830    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799833    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799835    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799838    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799841    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799844    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799846    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799849    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799851    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799854    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799857    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799859    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799862    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799864    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:42:35.803127 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799867    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799869    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799872    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799874    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799878    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799882    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799885    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799888    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799891    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799894    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799896    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799899    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799902    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799904    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799907    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799910    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799913    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799916    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799919    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:42:35.803646 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799923    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799926    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799929    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799932    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799937    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799941    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799944    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799947    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799950    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799952    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799955    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799958    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799960    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799963    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799966    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:42:35.804124 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.799969    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.799974    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800105    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800110    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800113    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800116    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800118    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800121    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800124    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800126    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800129    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800131    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800134    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800138    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800142    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:42:35.804510 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800145    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800148    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800150    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800153    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800156    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800158    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800161    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800164    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800166    2558 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800169    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800171    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800173    2558 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800177    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800179    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800181    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800184    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800186    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800189    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800191    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800194    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:42:35.804873 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800196    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800198    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800201    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800203    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800206    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800209    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800211    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800226    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800229    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800232    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800235    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800238    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800241    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800243    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800246    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800248    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800251    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800253    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800256    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800258    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:42:35.805370 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800261    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800264    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800267    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800269    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800272    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800275    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800278    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800280    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800283    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800286    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800288    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800291    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800293    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800297    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800300    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800303    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800306    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800309    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800312    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:42:35.805859 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800315    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800317    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800320    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800322    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800325    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800328    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800330    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800333    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800335    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800338    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800340    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800343    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800345    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800348    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800351    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800353    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800356    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800359    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800361    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:42:35.806326 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800364    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800368    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800371    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800374    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800377    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800379    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800382    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:35.800385    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.800389    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.800504    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.802977    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.804136    2558 server.go:1019] "Starting client certificate rotation"
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.804243    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:42:35.806794 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.804279    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:42:35.807114 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.807101    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:42:35.810211 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.810191    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:42:35.821729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.821705    2558 log.go:25] "Validated CRI v1 runtime API"
Sep 10 14:42:35.825085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.825067    2558 log.go:25] "Validated CRI v1 image API"
Sep 10 14:42:35.825926 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.825908    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 14:42:35.829988 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.829960    2558 fs.go:135] Filesystem UUIDs: map[5403e4ed-fceb-499c-8a97-eac0a2647db9:/dev/nvme0n1p4 62bc1e58-62c0-4a27-be4b-2cbcd1c6203c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 10 14:42:35.830066 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.829986    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 14:42:35.833102 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.833082    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:42:35.835434 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.835313    2558 manager.go:216] Machine: {Timestamp:2026-09-10 14:42:35.833779257 +0000 UTC m=+0.320467140 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101335 MemoryCapacity:32812146688 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25cc20ac89d2acb81e8af337438b05 SystemUUID:ec25cc20-ac89-d2ac-b81e-8af337438b05 BootID:e014573e-81ae-4b24-ade2-89dcd4f945f4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406073344 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:66:42:5b:ab:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:66:42:5b:ab:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:11:49:fe:8d:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812146688 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 14:42:35.835434 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.835432    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 14:42:35.835555 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.835525    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 14:42:35.839515 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.839487    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 14:42:35.839651 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.839519    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-253.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 14:42:35.840347 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.840337    2558 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 14:42:35.840384 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.840350    2558 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 14:42:35.840384 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.840358    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:42:35.841111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.841101    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:42:35.842774 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.842764    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 14:42:35.846847 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.846835    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:42:35.846975 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.846964    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 14:42:35.847015 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.847001    2558 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 14:42:35.847015 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.847014    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 14:42:35.847066 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.847029    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 14:42:35.847066 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.847039    2558 kubelet.go:398] "Adding apiserver pod source"
Sep 10 14:42:35.847066 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.847050    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 14:42:35.847437 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.847426    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:42:35.847478 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.847444    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:42:35.850031 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.850013    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 14:42:35.853235 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.853206    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 14:42:35.853292 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.853252    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 14:42:35.854249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854237    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 14:42:35.854284 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854257    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 14:42:35.854284 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854263    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 14:42:35.854284 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854269    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 14:42:35.854284 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854278    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 14:42:35.854284 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854284    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 14:42:35.854453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854297    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 14:42:35.854453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854302    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 14:42:35.854453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854311    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 14:42:35.854453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854316    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 14:42:35.854453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854324    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 14:42:35.854453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.854329    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 14:42:35.855115 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.855106    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 14:42:35.855154 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.855116    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 14:42:35.855197 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.855177    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ttls"
Sep 10 14:42:35.859578 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.859546    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-253.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 14:42:35.859658 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.859578    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 14:42:35.859658 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.859630    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-253.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 14:42:35.859766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.859712    2558 server.go:1268] "Started kubelet"
Sep 10 14:42:35.859817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.859769    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 14:42:35.859862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.859815    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 14:42:35.859910 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.859861    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 14:42:35.864439 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.864407    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ttls"
Sep 10 14:42:35.864530 ip-10-0-142-253 systemd[1]: Started Kubernetes Kubelet.
Sep 10 14:42:35.865047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.864983    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 14:42:35.866169 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.866149    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 14:42:35.866602 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.866581    2558 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 14:42:35.870253 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.869125    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-253.ec2.internal.18d3fcbab6eeb38a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-253.ec2.internal,UID:ip-10-0-142-253.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-253.ec2.internal,},FirstTimestamp:2026-09-10 14:42:35.85943233 +0000 UTC m=+0.346120215,LastTimestamp:2026-09-10 14:42:35.85943233 +0000 UTC m=+0.346120215,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-253.ec2.internal,}"
Sep 10 14:42:35.870953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.870937    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 14:42:35.871004 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.870955    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 14:42:35.871156 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.871129    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:35.871729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.871710    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 14:42:35.871729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.871718    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 14:42:35.871729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.871732    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 14:42:35.871867 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.871837    2558 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 14:42:35.871867 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.871847    2558 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 14:42:35.872649 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.872631    2558 factory.go:55] Registering systemd factory
Sep 10 14:42:35.872649 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.872652    2558 factory.go:223] Registration of the systemd container factory successfully
Sep 10 14:42:35.872877 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.872865    2558 factory.go:153] Registering CRI-O factory
Sep 10 14:42:35.872877 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.872876    2558 factory.go:223] Registration of the crio container factory successfully
Sep 10 14:42:35.872949 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.872922    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 14:42:35.872949 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.872942    2558 factory.go:103] Registering Raw factory
Sep 10 14:42:35.873001 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.872954    2558 manager.go:1196] Started watching for new ooms in manager
Sep 10 14:42:35.873318 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.873305    2558 manager.go:349] Starting recovery of all containers
Sep 10 14:42:35.881439 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.881417    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:42:35.882930 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.882914    2558 manager.go:354] Recovery completed
Sep 10 14:42:35.883785 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.883758    2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-253.ec2.internal\" not found" node="ip-10-0-142-253.ec2.internal"
Sep 10 14:42:35.884672 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.884606    2558 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 10 14:42:35.887771 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.887758    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:42:35.889799 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.889786    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:42:35.889859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.889812    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:42:35.889859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.889822    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:42:35.890449 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.890436    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 14:42:35.890497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.890449    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 14:42:35.890497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.890456    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:42:35.890497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.890469    2558 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:42:35.891537 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.891524    2558 policy_none.go:49] "None policy: Start"
Sep 10 14:42:35.891607 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.891542    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 14:42:35.891607 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.891570    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 14:42:35.891678 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.891604    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:42:35.891678 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.891638    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:42:35.891678 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.891650    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:42:35.892693 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.892682    2558 policy_none.go:47] "Start"
Sep 10 14:42:35.934873 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.934855    2558 manager.go:337] "Starting Device Plugin manager"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.934901    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.934913    2558 server.go:87] "Starting device plugin registration server"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.935204    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.935235    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.935340    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.935437    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:35.935448    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.936063    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 14:42:35.944862 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:35.936107    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.010452 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.010410    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 14:42:36.011844 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.011811    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 14:42:36.011949 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.011862    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 14:42:36.011949 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.011888    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 14:42:36.011949 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.011926    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 14:42:36.013490 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.013468    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:42:36.036124 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.036023    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:42:36.036999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.036982    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:42:36.037077 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.037015    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:42:36.037077 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.037029    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:42:36.037077 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.037050    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.044099 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.044083    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.044150 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.044107    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-253.ec2.internal\": node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.061824 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.061801    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.112584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.112554    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal"]
Sep 10 14:42:36.112658 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.112634    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:42:36.115529 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.115509    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:42:36.115620 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.115549    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:42:36.115620 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.115564    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:42:36.118061 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118044    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:42:36.118173 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.118256 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118189    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:42:36.118819 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118802    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:42:36.118888 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118809    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:42:36.118888 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118844    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:42:36.118888 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118855    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:42:36.118992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118828    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:42:36.118992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.118916    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:42:36.120353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.120339    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.120397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.120367    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:42:36.120996 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.120982    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:42:36.121081 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.121009    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:42:36.121081 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.121022    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:42:36.143945 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.143923    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-253.ec2.internal\" not found" node="ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.148348 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.148334    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-253.ec2.internal\" not found" node="ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.162433 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.162401    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.263291 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.263257    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.272580 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.272554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2d6563cf27d06d8ea6209c65b6252bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal\" (UID: \"d2d6563cf27d06d8ea6209c65b6252bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.272672 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.272588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2d6563cf27d06d8ea6209c65b6252bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal\" (UID: \"d2d6563cf27d06d8ea6209c65b6252bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.272672 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.272610    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ff70c42461e7f0c1049ab3f77cb139d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-253.ec2.internal\" (UID: \"6ff70c42461e7f0c1049ab3f77cb139d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.364024 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.363958    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.373253 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.373205    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2d6563cf27d06d8ea6209c65b6252bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal\" (UID: \"d2d6563cf27d06d8ea6209c65b6252bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.373334 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.373264    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2d6563cf27d06d8ea6209c65b6252bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal\" (UID: \"d2d6563cf27d06d8ea6209c65b6252bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.373334 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.373291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ff70c42461e7f0c1049ab3f77cb139d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-253.ec2.internal\" (UID: \"6ff70c42461e7f0c1049ab3f77cb139d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.373427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.373335    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2d6563cf27d06d8ea6209c65b6252bc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal\" (UID: \"d2d6563cf27d06d8ea6209c65b6252bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.373427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.373339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ff70c42461e7f0c1049ab3f77cb139d-config\") pod \"kube-apiserver-proxy-ip-10-0-142-253.ec2.internal\" (UID: \"6ff70c42461e7f0c1049ab3f77cb139d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.373427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.373335    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2d6563cf27d06d8ea6209c65b6252bc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal\" (UID: \"d2d6563cf27d06d8ea6209c65b6252bc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.446404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.446372    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.451381 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.451247    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:36.464517 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.464494    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.565022 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.564978    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.665530 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.665464    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.766139 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.766113    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.803619 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.803599    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 14:42:36.804022 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.803725    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 14:42:36.804022 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.803752    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 14:42:36.867075 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.867046    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:36.870138 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.870107    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 14:37:36 +0000 UTC" deadline="2028-06-23 09:17:36.041777534 +0000 UTC"
Sep 10 14:42:36.870188 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:36.870138    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15642h34m59.171642123s"
Sep 10 14:42:36.967374 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:36.967342    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:37.054388 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.054361    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:42:37.068486 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.068435    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:37.079441 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:37.079414    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d6563cf27d06d8ea6209c65b6252bc.slice/crio-ac8ee9dd128538821722fbfa8d5f1e6bbb35a5ff832f6462ec14f76491359257 WatchSource:0}: Error finding container ac8ee9dd128538821722fbfa8d5f1e6bbb35a5ff832f6462ec14f76491359257: Status 404 returned error can't find the container with id ac8ee9dd128538821722fbfa8d5f1e6bbb35a5ff832f6462ec14f76491359257
Sep 10 14:42:37.079622 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:37.079611    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff70c42461e7f0c1049ab3f77cb139d.slice/crio-adbcae34c0cad7e019529ce51ac175e53f7b737a550a8f1575c722f000c61399 WatchSource:0}: Error finding container adbcae34c0cad7e019529ce51ac175e53f7b737a550a8f1575c722f000c61399: Status 404 returned error can't find the container with id adbcae34c0cad7e019529ce51ac175e53f7b737a550a8f1575c722f000c61399
Sep 10 14:42:37.083402 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.083382    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:42:37.169559 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.169523    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:37.270071 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.269995    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-253.ec2.internal\" not found"
Sep 10 14:42:37.347601 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.347572    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:42:37.371111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.371088    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:37.381804 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.381783    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:42:37.382973 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.382952    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal"
Sep 10 14:42:37.390912 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.390888    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:42:37.848142 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.848108    2558 apiserver.go:52] "Watching apiserver"
Sep 10 14:42:37.859236 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.859199    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 14:42:37.862429 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.861113    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-g7dg8","kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal","openshift-image-registry/node-ca-nmjz6","openshift-multus/multus-zfx7h","openshift-network-diagnostics/network-check-target-jt5pz","openshift-ovn-kubernetes/ovnkube-node-rjdrt","kube-system/konnectivity-agent-vp8pc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv","openshift-cluster-node-tuning-operator/tuned-8bp45","openshift-dns/node-resolver-qglhm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal","openshift-multus/multus-additional-cni-plugins-8nc57","openshift-multus/network-metrics-daemon-hbxnv","openshift-network-operator/iptables-alerter-2tvr6"]
Sep 10 14:42:37.865175 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.863802    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:37.865175 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.864944    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:37.865604 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.865125    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:37.866293 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.866270    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 14:42:37.866890 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.866716    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.871039 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.870440    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hlwsw\""
Sep 10 14:42:37.871039 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.870643    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 14:42:37.871185 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.871071    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 14:42:37.871264 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.871250    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qhscm\""
Sep 10 14:42:37.872093 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.871718    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 14:42:37.872093 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.871914    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 14:42:37.872449 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.872426    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 14:42:37.873952 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.873290    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.874873 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.874852    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:37.874965 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.874918    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:37.875975 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.875692    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 14:42:37.875975 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.875800    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 14:42:37.876641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.876225    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 14:42:37.876641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.876321    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.876641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.876360    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 14:42:37.876641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.876473    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wsft2\""
Sep 10 14:42:37.877866 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.877848    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.879109 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.879083    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 14:42:37.879212 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.879118    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 14:42:37.879350 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.879296    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 14:42:37.879415 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.879383    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:37.879472 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.879446    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:37.880511 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.880156    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 14:42:37.880511 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.880180    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jqkh4\""
Sep 10 14:42:37.880511 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.880268    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 14:42:37.880511 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.880320    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:42:37.880511 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.880502    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 14:42:37.880818 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.880698    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qvtnm\""
Sep 10 14:42:37.880978 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.880961    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:37.883077 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.882300    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:37.883077 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.882534    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 14:42:37.883232 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883134    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-multus-certs\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883232 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883169    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-system-cni-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883232 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883198    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-cnibin\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883422 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883225    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d36866a7-ce48-4c5f-b14a-b3cad558f061-cni-binary-copy\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883422 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883368    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:37.883422 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883397    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d4c95f-8587-43ee-addb-325df5853d41-serviceca\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.883579 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-netns\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883579 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-cni-multus\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883579 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883500    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-daemon-config\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883579 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883532    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-etc-kubernetes\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883579 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883558    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnbg7\" (UniqueName: \"kubernetes.io/projected/d36866a7-ce48-4c5f-b14a-b3cad558f061-kube-api-access-gnbg7\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883606    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de043964-4505-4d6c-9af0-62ba8b338a9d-konnectivity-ca\") pod \"konnectivity-agent-vp8pc\" (UID: \"de043964-4505-4d6c-9af0-62ba8b338a9d\") " pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:37.883832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stsn6\" (UniqueName: \"kubernetes.io/projected/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-kube-api-access-stsn6\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:37.883832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883663    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d4c95f-8587-43ee-addb-325df5853d41-host\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.883832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883693    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjb2b\" (UniqueName: \"kubernetes.io/projected/11d4c95f-8587-43ee-addb-325df5853d41-kube-api-access-fjb2b\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.883832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-cni-bin\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.883832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883738    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de043964-4505-4d6c-9af0-62ba8b338a9d-agent-certs\") pod \"konnectivity-agent-vp8pc\" (UID: \"de043964-4505-4d6c-9af0-62ba8b338a9d\") " pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:37.883832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-cni-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.884166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883846    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-socket-dir-parent\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.884166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883874    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-k8s-cni-cncf-io\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.884166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883909    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-kubelet\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.884166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883940    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-hostroot\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.884166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.883965    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-conf-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.884493 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.884191    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.885037 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.885018    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:42:37.885227 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.885209    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 14:42:37.885455 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.885439    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 14:42:37.885702 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.885685    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 14:42:37.885793 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.885707    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jxjhx\""
Sep 10 14:42:37.885849 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.885824    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9p9w6\""
Sep 10 14:42:37.885967 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.885943    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.886133 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.886041    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 14:42:37.886133 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.886048    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:42:37.886599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.886558    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 14:42:37.886820 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.886805    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 14:42:37.888339 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.887286    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 14:42:37.888339 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.887833    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7t6xn\""
Sep 10 14:42:37.889480 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.888731    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 14:42:37.889480 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.888885    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q2tx4\""
Sep 10 14:42:37.889480 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.889189    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 14:42:37.904709 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.904212    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d4cq6"
Sep 10 14:42:37.914316 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.912791    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d4cq6"
Sep 10 14:42:37.960237 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.960212    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:42:37.973193 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.973169    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 14:42:37.984826 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984798    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.984956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984835    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysctl-conf\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.984956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984862    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-registration-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.984956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984887    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-etc-selinux\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.984956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de043964-4505-4d6c-9af0-62ba8b338a9d-agent-certs\") pod \"konnectivity-agent-vp8pc\" (UID: \"de043964-4505-4d6c-9af0-62ba8b338a9d\") " pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:37.984956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984940    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-k8s-cni-cncf-io\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-hostroot\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.984989    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysctl-d\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-sys\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985036    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-var-lib-kubelet\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985059    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbb51f27-bb88-427d-8e1b-86bd53b34d73-tmp\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985083    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bcfb93f7-db68-46f7-8445-b9c768f63b8a-iptables-alerter-script\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985111    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-cni-multus\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985135    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-kubelet\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985161    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-cni-bin\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985186    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgzg7\" (UniqueName: \"kubernetes.io/projected/cbb51f27-bb88-427d-8e1b-86bd53b34d73-kube-api-access-kgzg7\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.985225 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb4rr\" (UniqueName: \"kubernetes.io/projected/bcfb93f7-db68-46f7-8445-b9c768f63b8a-kube-api-access-nb4rr\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg696\" (UniqueName: \"kubernetes.io/projected/523956a8-c95c-44d7-8a58-48997ea2f395-kube-api-access-hg696\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985300    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-etc-kubernetes\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985356    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/523956a8-c95c-44d7-8a58-48997ea2f395-hosts-file\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de043964-4505-4d6c-9af0-62ba8b338a9d-konnectivity-ca\") pod \"konnectivity-agent-vp8pc\" (UID: \"de043964-4505-4d6c-9af0-62ba8b338a9d\") " pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985411    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stsn6\" (UniqueName: \"kubernetes.io/projected/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-kube-api-access-stsn6\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985439    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985465    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-system-cni-dir\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985493    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985531    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-socket-dir-parent\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985556    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-kubelet\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-etc-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-ovnkube-script-lib\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985640    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-socket-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985668    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-device-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.985814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-826ln\" (UniqueName: \"kubernetes.io/projected/0a44e73c-8b44-4d88-97b0-ce0446954353-kube-api-access-826ln\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-system-cni-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985778    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985804    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-netns\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985830    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.985883    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-env-overrides\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986054    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcfb93f7-db68-46f7-8445-b9c768f63b8a-host-slash\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986076    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-system-cni-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986084    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986112    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-sys-fs\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986142    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-cni-binary-copy\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d4c95f-8587-43ee-addb-325df5853d41-host\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986198    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-cni-bin\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986229    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysconfig\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-host\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986251    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-socket-dir-parent\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986283    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-multus-certs\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.986629 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-modprobe-d\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1938126-5194-491b-b8e3-a602e268c691-dbus\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d36866a7-ce48-4c5f-b14a-b3cad558f061-cni-binary-copy\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d4c95f-8587-43ee-addb-325df5853d41-serviceca\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnbg7\" (UniqueName: \"kubernetes.io/projected/d36866a7-ce48-4c5f-b14a-b3cad558f061-kube-api-access-gnbg7\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986475    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21e67658-4290-47db-ae99-95df1ceebb2d-ovn-node-metrics-cert\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt6jz\" (UniqueName: \"kubernetes.io/projected/71c85297-913a-4b34-86e4-83f1acb6b631-kube-api-access-lt6jz\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-cnibin\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986573    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-systemd\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986598    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-var-lib-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-cni-netd\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986653    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjb2b\" (UniqueName: \"kubernetes.io/projected/11d4c95f-8587-43ee-addb-325df5853d41-kube-api-access-fjb2b\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986682    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6cdj\" (UniqueName: \"kubernetes.io/projected/21e67658-4290-47db-ae99-95df1ceebb2d-kube-api-access-k6cdj\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986708    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-lib-modules\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986734    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1938126-5194-491b-b8e3-a602e268c691-kubelet-config\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986770    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/523956a8-c95c-44d7-8a58-48997ea2f395-tmp-dir\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-cni-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.986995    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-conf-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987024    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-log-socket\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987053    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-ovnkube-config\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-kubernetes\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987112    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-cnibin\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987138    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-daemon-config\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-systemd-units\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987250    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-ovn\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987259    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987277    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-node-log\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-systemd\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987352    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-tuned\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987389    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987417    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-slash\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987441    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-run\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987469    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987510    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:37.988411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987544    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-run-netns\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-cni-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987749    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-kubelet\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987785    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-cni-multus\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.987840    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-netns\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.987920    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:37.988007    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs podName:0fb75b7e-b7c0-4bfa-8c51-9b96e4449391 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:38.487964552 +0000 UTC m=+2.974652433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs") pod "network-metrics-daemon-hbxnv" (UID: "0fb75b7e-b7c0-4bfa-8c51-9b96e4449391") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.988233    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d4c95f-8587-43ee-addb-325df5853d41-host\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.988305    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-var-lib-cni-bin\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.988351    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-multus-certs\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.988391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-host-run-k8s-cni-cncf-io\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.988480    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-etc-kubernetes\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.988524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-hostroot\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.988931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d36866a7-ce48-4c5f-b14a-b3cad558f061-cni-binary-copy\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.989051    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-cnibin\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989977 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.989374    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d4c95f-8587-43ee-addb-325df5853d41-serviceca\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:37.989977 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.989571    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-daemon-config\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.989977 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.989863    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de043964-4505-4d6c-9af0-62ba8b338a9d-konnectivity-ca\") pod \"konnectivity-agent-vp8pc\" (UID: \"de043964-4505-4d6c-9af0-62ba8b338a9d\") " pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:37.989977 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.989930    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d36866a7-ce48-4c5f-b14a-b3cad558f061-multus-conf-dir\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:37.996403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.996344    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de043964-4505-4d6c-9af0-62ba8b338a9d-agent-certs\") pod \"konnectivity-agent-vp8pc\" (UID: \"de043964-4505-4d6c-9af0-62ba8b338a9d\") " pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:37.998865 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:37.998835    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjb2b\" (UniqueName: \"kubernetes.io/projected/11d4c95f-8587-43ee-addb-325df5853d41-kube-api-access-fjb2b\") pod \"node-ca-nmjz6\" (UID: \"11d4c95f-8587-43ee-addb-325df5853d41\") " pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:38.000951 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.000929    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stsn6\" (UniqueName: \"kubernetes.io/projected/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-kube-api-access-stsn6\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:38.001233 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.001212    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnbg7\" (UniqueName: \"kubernetes.io/projected/d36866a7-ce48-4c5f-b14a-b3cad558f061-kube-api-access-gnbg7\") pod \"multus-zfx7h\" (UID: \"d36866a7-ce48-4c5f-b14a-b3cad558f061\") " pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:38.019284 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.019212    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal" event={"ID":"d2d6563cf27d06d8ea6209c65b6252bc","Type":"ContainerStarted","Data":"ac8ee9dd128538821722fbfa8d5f1e6bbb35a5ff832f6462ec14f76491359257"}
Sep 10 14:42:38.020726 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.020700    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal" event={"ID":"6ff70c42461e7f0c1049ab3f77cb139d","Type":"ContainerStarted","Data":"adbcae34c0cad7e019529ce51ac175e53f7b737a550a8f1575c722f000c61399"}
Sep 10 14:42:38.088312 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088266    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.088312 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088318    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-sys-fs\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088347    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-cni-binary-copy\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088371    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysconfig\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088387    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-host\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-modprobe-d\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088432    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-sys-fs\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysconfig\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088483    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-host\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088528    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1938126-5194-491b-b8e3-a602e268c691-dbus\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:38.088561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088549    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21e67658-4290-47db-ae99-95df1ceebb2d-ovn-node-metrics-cert\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1938126-5194-491b-b8e3-a602e268c691-dbus\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-modprobe-d\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088701    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt6jz\" (UniqueName: \"kubernetes.io/projected/71c85297-913a-4b34-86e4-83f1acb6b631-kube-api-access-lt6jz\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-cnibin\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088789    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-systemd\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-var-lib-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088831    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-cnibin\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088881    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-systemd\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088841    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-cni-netd\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088901    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-var-lib-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088888    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-cni-netd\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088924    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6cdj\" (UniqueName: \"kubernetes.io/projected/21e67658-4290-47db-ae99-95df1ceebb2d-kube-api-access-k6cdj\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088952    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-lib-modules\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.088977    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1938126-5194-491b-b8e3-a602e268c691-kubelet-config\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089001    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/523956a8-c95c-44d7-8a58-48997ea2f395-tmp-dir\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-log-socket\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089050    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-ovnkube-config\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-kubernetes\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089104    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-systemd-units\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089104    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1938126-5194-491b-b8e3-a602e268c691-kubelet-config\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089128    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-cni-binary-copy\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089127    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-ovn\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089177    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-ovn\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-node-log\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089218    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-systemd\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089227    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-node-log\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-tuned\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089273    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-systemd-units\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089327    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-slash\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089350    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-run\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089354    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-kubernetes\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089373    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:38.089968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089399    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089425    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-run-netns\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089449    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089472    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysctl-conf\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089485    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/523956a8-c95c-44d7-8a58-48997ea2f395-tmp-dir\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089498    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-registration-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089508    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089523    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-etc-selinux\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089552    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysctl-d\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089560    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-systemd\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089562    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-log-socket\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-sys\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089614    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-var-lib-kubelet\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089624    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-sys\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089641    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbb51f27-bb88-427d-8e1b-86bd53b34d73-tmp\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089666    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-slash\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089679    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-ovnkube-config\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089718    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-var-lib-kubelet\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.090730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089720    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-run\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089763    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-registration-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089836    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysctl-conf\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-etc-selinux\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-lib-modules\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.089923    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-run-netns\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.089994    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret podName:c1938126-5194-491b-b8e3-a602e268c691 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:38.589974433 +0000 UTC m=+3.076662311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret") pod "global-pull-secret-syncer-g7dg8" (UID: "c1938126-5194-491b-b8e3-a602e268c691") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089990    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-sysctl-d\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.089668    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bcfb93f7-db68-46f7-8445-b9c768f63b8a-iptables-alerter-script\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-run-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-kubelet\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-cni-bin\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgzg7\" (UniqueName: \"kubernetes.io/projected/cbb51f27-bb88-427d-8e1b-86bd53b34d73-kube-api-access-kgzg7\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb4rr\" (UniqueName: \"kubernetes.io/projected/bcfb93f7-db68-46f7-8445-b9c768f63b8a-kube-api-access-nb4rr\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg696\" (UniqueName: \"kubernetes.io/projected/523956a8-c95c-44d7-8a58-48997ea2f395-kube-api-access-hg696\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.091641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090200    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/523956a8-c95c-44d7-8a58-48997ea2f395-hosts-file\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090254    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-system-cni-dir\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090280    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090319    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bcfb93f7-db68-46f7-8445-b9c768f63b8a-iptables-alerter-script\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-etc-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-ovnkube-script-lib\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090406    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/523956a8-c95c-44d7-8a58-48997ea2f395-hosts-file\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-socket-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-device-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090465    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-826ln\" (UniqueName: \"kubernetes.io/projected/0a44e73c-8b44-4d88-97b0-ce0446954353-kube-api-access-826ln\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090466    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-run-ovn-kubernetes\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090472    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090517    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090544    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-env-overrides\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcfb93f7-db68-46f7-8445-b9c768f63b8a-host-slash\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090643    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcfb93f7-db68-46f7-8445-b9c768f63b8a-host-slash\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:38.092517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-kubelet\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090689    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-etc-openvswitch\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090732    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-cni-bin\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090762    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21e67658-4290-47db-ae99-95df1ceebb2d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090854    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-socket-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71c85297-913a-4b34-86e4-83f1acb6b631-device-dir\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090926    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a44e73c-8b44-4d88-97b0-ce0446954353-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.090947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a44e73c-8b44-4d88-97b0-ce0446954353-system-cni-dir\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.091172    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-ovnkube-script-lib\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.093397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.091263    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21e67658-4290-47db-ae99-95df1ceebb2d-env-overrides\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.094525 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.094496    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbb51f27-bb88-427d-8e1b-86bd53b34d73-tmp\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.094640 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.094536    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cbb51f27-bb88-427d-8e1b-86bd53b34d73-etc-tuned\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.094640 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.094567    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21e67658-4290-47db-ae99-95df1ceebb2d-ovn-node-metrics-cert\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.097072 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.097043    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt6jz\" (UniqueName: \"kubernetes.io/projected/71c85297-913a-4b34-86e4-83f1acb6b631-kube-api-access-lt6jz\") pod \"aws-ebs-csi-driver-node-4bqsv\" (UID: \"71c85297-913a-4b34-86e4-83f1acb6b631\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.097371 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.097347    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6cdj\" (UniqueName: \"kubernetes.io/projected/21e67658-4290-47db-ae99-95df1ceebb2d-kube-api-access-k6cdj\") pod \"ovnkube-node-rjdrt\" (UID: \"21e67658-4290-47db-ae99-95df1ceebb2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.098998 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.098977    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:42:38.099099 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.099002    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:42:38.099099 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.099016    2558 projected.go:196] Error preparing data for projected volume kube-api-access-s6hpv for pod openshift-network-diagnostics/network-check-target-jt5pz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:38.099099 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.099085    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv podName:f0076e40-95a9-473a-bfdc-2e93c30e9f7d nodeName:}" failed. No retries permitted until 2026-09-10 14:42:38.59906759 +0000 UTC m=+3.085755467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s6hpv" (UniqueName: "kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv") pod "network-check-target-jt5pz" (UID: "f0076e40-95a9-473a-bfdc-2e93c30e9f7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:38.101394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.101364    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgzg7\" (UniqueName: \"kubernetes.io/projected/cbb51f27-bb88-427d-8e1b-86bd53b34d73-kube-api-access-kgzg7\") pod \"tuned-8bp45\" (UID: \"cbb51f27-bb88-427d-8e1b-86bd53b34d73\") " pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.101708 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.101687    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb4rr\" (UniqueName: \"kubernetes.io/projected/bcfb93f7-db68-46f7-8445-b9c768f63b8a-kube-api-access-nb4rr\") pod \"iptables-alerter-2tvr6\" (UID: \"bcfb93f7-db68-46f7-8445-b9c768f63b8a\") " pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:38.102357 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.102340    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg696\" (UniqueName: \"kubernetes.io/projected/523956a8-c95c-44d7-8a58-48997ea2f395-kube-api-access-hg696\") pod \"node-resolver-qglhm\" (UID: \"523956a8-c95c-44d7-8a58-48997ea2f395\") " pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:38.102637 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.102614    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-826ln\" (UniqueName: \"kubernetes.io/projected/0a44e73c-8b44-4d88-97b0-ce0446954353-kube-api-access-826ln\") pod \"multus-additional-cni-plugins-8nc57\" (UID: \"0a44e73c-8b44-4d88-97b0-ce0446954353\") " pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.183704 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.183605    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:42:38.192506 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.192475    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nmjz6"
Sep 10 14:42:38.206494 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.206468    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zfx7h"
Sep 10 14:42:38.212543 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.212520    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:42:38.227453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.227424    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8bp45"
Sep 10 14:42:38.230943 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.230919    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:42:38.238221 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.238196    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2tvr6"
Sep 10 14:42:38.245182 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.245157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qglhm"
Sep 10 14:42:38.255307 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.255283    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv"
Sep 10 14:42:38.263191 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.263167    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8nc57"
Sep 10 14:42:38.494641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.494552    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:38.494795 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.494701    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:38.494847 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.494827    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs podName:0fb75b7e-b7c0-4bfa-8c51-9b96e4449391 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:39.494803114 +0000 UTC m=+3.981491006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs") pod "network-metrics-daemon-hbxnv" (UID: "0fb75b7e-b7c0-4bfa-8c51-9b96e4449391") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:38.595716 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.595676    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:38.595902 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.595841    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:38.595976 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.595909    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret podName:c1938126-5194-491b-b8e3-a602e268c691 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:39.595891118 +0000 UTC m=+4.082578996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret") pod "global-pull-secret-syncer-g7dg8" (UID: "c1938126-5194-491b-b8e3-a602e268c691") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:38.696313 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.696274    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:38.696496 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.696434    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:42:38.696496 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.696453    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:42:38.696496 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.696462    2558 projected.go:196] Error preparing data for projected volume kube-api-access-s6hpv for pod openshift-network-diagnostics/network-check-target-jt5pz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:38.696624 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:38.696535    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv podName:f0076e40-95a9-473a-bfdc-2e93c30e9f7d nodeName:}" failed. No retries permitted until 2026-09-10 14:42:39.696514356 +0000 UTC m=+4.183202252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s6hpv" (UniqueName: "kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv") pod "network-check-target-jt5pz" (UID: "f0076e40-95a9-473a-bfdc-2e93c30e9f7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:38.915788 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.915699    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:37:38 +0000 UTC" deadline="2028-02-29 09:53:57.782682228 +0000 UTC"
Sep 10 14:42:38.915788 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:38.915733    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12883h11m18.866952628s"
Sep 10 14:42:39.012160 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:39.012128    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:39.012342 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.012276    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:39.097084 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.097052    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e67658_4290_47db_ae99_95df1ceebb2d.slice/crio-f9cb0f1bb870bd099022166c7d7ad06c09c6a654adcb7d093f396bb29564f0c1 WatchSource:0}: Error finding container f9cb0f1bb870bd099022166c7d7ad06c09c6a654adcb7d093f396bb29564f0c1: Status 404 returned error can't find the container with id f9cb0f1bb870bd099022166c7d7ad06c09c6a654adcb7d093f396bb29564f0c1
Sep 10 14:42:39.098098 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.098053    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36866a7_ce48_4c5f_b14a_b3cad558f061.slice/crio-e36cc592cd80b4b260dcdaf183410c4f384786e672cbc904e225ffc37aa3f87d WatchSource:0}: Error finding container e36cc592cd80b4b260dcdaf183410c4f384786e672cbc904e225ffc37aa3f87d: Status 404 returned error can't find the container with id e36cc592cd80b4b260dcdaf183410c4f384786e672cbc904e225ffc37aa3f87d
Sep 10 14:42:39.100685 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.100505    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfb93f7_db68_46f7_8445_b9c768f63b8a.slice/crio-df03dac5d4a5fedb185d9ac2db9b46f83eee3b21790d42f11abad2406910a543 WatchSource:0}: Error finding container df03dac5d4a5fedb185d9ac2db9b46f83eee3b21790d42f11abad2406910a543: Status 404 returned error can't find the container with id df03dac5d4a5fedb185d9ac2db9b46f83eee3b21790d42f11abad2406910a543
Sep 10 14:42:39.103689 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.103492    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb51f27_bb88_427d_8e1b_86bd53b34d73.slice/crio-87b15aaf95449fbd6d17570ae676455460984cffb916e27adc2996805960dcbd WatchSource:0}: Error finding container 87b15aaf95449fbd6d17570ae676455460984cffb916e27adc2996805960dcbd: Status 404 returned error can't find the container with id 87b15aaf95449fbd6d17570ae676455460984cffb916e27adc2996805960dcbd
Sep 10 14:42:39.105466 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.105043    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod523956a8_c95c_44d7_8a58_48997ea2f395.slice/crio-e109b1e2bc8302f01456ac63c5b80ecfe346c648f3a365a92ef2780cda958050 WatchSource:0}: Error finding container e109b1e2bc8302f01456ac63c5b80ecfe346c648f3a365a92ef2780cda958050: Status 404 returned error can't find the container with id e109b1e2bc8302f01456ac63c5b80ecfe346c648f3a365a92ef2780cda958050
Sep 10 14:42:39.106519 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.106490    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d4c95f_8587_43ee_addb_325df5853d41.slice/crio-e80a68033802b2668f6d7c8bc43ff44a5c05ed1afd1d3cb29bc6b459e0f8e28e WatchSource:0}: Error finding container e80a68033802b2668f6d7c8bc43ff44a5c05ed1afd1d3cb29bc6b459e0f8e28e: Status 404 returned error can't find the container with id e80a68033802b2668f6d7c8bc43ff44a5c05ed1afd1d3cb29bc6b459e0f8e28e
Sep 10 14:42:39.107731 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.107648    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a44e73c_8b44_4d88_97b0_ce0446954353.slice/crio-20df10f7a2ac5631801afe9ea130d91d9d046bc68338d53dbbdaebb554f2f3ce WatchSource:0}: Error finding container 20df10f7a2ac5631801afe9ea130d91d9d046bc68338d53dbbdaebb554f2f3ce: Status 404 returned error can't find the container with id 20df10f7a2ac5631801afe9ea130d91d9d046bc68338d53dbbdaebb554f2f3ce
Sep 10 14:42:39.108051 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.107935    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c85297_913a_4b34_86e4_83f1acb6b631.slice/crio-eee85134dd014ab1428352f0706212e630cea286a1d724436f23ef9f7fb2d186 WatchSource:0}: Error finding container eee85134dd014ab1428352f0706212e630cea286a1d724436f23ef9f7fb2d186: Status 404 returned error can't find the container with id eee85134dd014ab1428352f0706212e630cea286a1d724436f23ef9f7fb2d186
Sep 10 14:42:39.109708 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:42:39.109381    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde043964_4505_4d6c_9af0_62ba8b338a9d.slice/crio-e94adf9bf665cffa18334cfe21544f21d3d80e30cc7703c766194c7f92913d70 WatchSource:0}: Error finding container e94adf9bf665cffa18334cfe21544f21d3d80e30cc7703c766194c7f92913d70: Status 404 returned error can't find the container with id e94adf9bf665cffa18334cfe21544f21d3d80e30cc7703c766194c7f92913d70
Sep 10 14:42:39.503012 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:39.502717    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:39.503167 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.503115    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:39.503224 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.503186    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs podName:0fb75b7e-b7c0-4bfa-8c51-9b96e4449391 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:41.503165557 +0000 UTC m=+5.989853434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs") pod "network-metrics-daemon-hbxnv" (UID: "0fb75b7e-b7c0-4bfa-8c51-9b96e4449391") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:39.603831 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:39.603788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:39.604082 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.604015    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:39.604153 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.604084    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret podName:c1938126-5194-491b-b8e3-a602e268c691 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:41.604064773 +0000 UTC m=+6.090752650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret") pod "global-pull-secret-syncer-g7dg8" (UID: "c1938126-5194-491b-b8e3-a602e268c691") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:39.705950 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:39.705265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:39.705950 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.705481    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:42:39.705950 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.705502    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:42:39.705950 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.705515    2558 projected.go:196] Error preparing data for projected volume kube-api-access-s6hpv for pod openshift-network-diagnostics/network-check-target-jt5pz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:39.705950 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:39.705576    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv podName:f0076e40-95a9-473a-bfdc-2e93c30e9f7d nodeName:}" failed. No retries permitted until 2026-09-10 14:42:41.705557442 +0000 UTC m=+6.192245327 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s6hpv" (UniqueName: "kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv") pod "network-check-target-jt5pz" (UID: "f0076e40-95a9-473a-bfdc-2e93c30e9f7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:39.916645 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:39.916563    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:37:38 +0000 UTC" deadline="2028-06-01 19:56:30.67947212 +0000 UTC"
Sep 10 14:42:39.916645 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:39.916638    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15125h13m50.762838501s"
Sep 10 14:42:40.015108 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.015074    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:40.015256 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:40.015212    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:40.015685 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.015662    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:40.015807 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:40.015785    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:40.029816 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.029782    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"f9cb0f1bb870bd099022166c7d7ad06c09c6a654adcb7d093f396bb29564f0c1"}
Sep 10 14:42:40.045710 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.045678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal" event={"ID":"6ff70c42461e7f0c1049ab3f77cb139d","Type":"ContainerStarted","Data":"eb51b643551ca092b8c122e2de6b086625d5be48cbbfb17cf3738e46341c7e38"}
Sep 10 14:42:40.055999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.055972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv" event={"ID":"71c85297-913a-4b34-86e4-83f1acb6b631","Type":"ContainerStarted","Data":"eee85134dd014ab1428352f0706212e630cea286a1d724436f23ef9f7fb2d186"}
Sep 10 14:42:40.062033 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.062000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerStarted","Data":"20df10f7a2ac5631801afe9ea130d91d9d046bc68338d53dbbdaebb554f2f3ce"}
Sep 10 14:42:40.064404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.064374    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8bp45" event={"ID":"cbb51f27-bb88-427d-8e1b-86bd53b34d73","Type":"ContainerStarted","Data":"87b15aaf95449fbd6d17570ae676455460984cffb916e27adc2996805960dcbd"}
Sep 10 14:42:40.067167 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.067137    2558 generic.go:356] "Generic (PLEG): container finished" podID="d2d6563cf27d06d8ea6209c65b6252bc" containerID="6c798264c380ae6064ce115f76ebab552420941675401f22b5edef0364ad3998" exitCode=0
Sep 10 14:42:40.067281 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.067222    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal" event={"ID":"d2d6563cf27d06d8ea6209c65b6252bc","Type":"ContainerDied","Data":"6c798264c380ae6064ce115f76ebab552420941675401f22b5edef0364ad3998"}
Sep 10 14:42:40.079102 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.079072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vp8pc" event={"ID":"de043964-4505-4d6c-9af0-62ba8b338a9d","Type":"ContainerStarted","Data":"e94adf9bf665cffa18334cfe21544f21d3d80e30cc7703c766194c7f92913d70"}
Sep 10 14:42:40.086504 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.086448    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-253.ec2.internal" podStartSLOduration=3.086428691 podStartE2EDuration="3.086428691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:40.061357012 +0000 UTC m=+4.548044910" watchObservedRunningTime="2026-09-10 14:42:40.086428691 +0000 UTC m=+4.573116578"
Sep 10 14:42:40.098055 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.098026    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nmjz6" event={"ID":"11d4c95f-8587-43ee-addb-325df5853d41","Type":"ContainerStarted","Data":"e80a68033802b2668f6d7c8bc43ff44a5c05ed1afd1d3cb29bc6b459e0f8e28e"}
Sep 10 14:42:40.105476 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.105439    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qglhm" event={"ID":"523956a8-c95c-44d7-8a58-48997ea2f395","Type":"ContainerStarted","Data":"e109b1e2bc8302f01456ac63c5b80ecfe346c648f3a365a92ef2780cda958050"}
Sep 10 14:42:40.131380 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.131342    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zfx7h" event={"ID":"d36866a7-ce48-4c5f-b14a-b3cad558f061","Type":"ContainerStarted","Data":"e36cc592cd80b4b260dcdaf183410c4f384786e672cbc904e225ffc37aa3f87d"}
Sep 10 14:42:40.160818 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:40.160690    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2tvr6" event={"ID":"bcfb93f7-db68-46f7-8445-b9c768f63b8a","Type":"ContainerStarted","Data":"df03dac5d4a5fedb185d9ac2db9b46f83eee3b21790d42f11abad2406910a543"}
Sep 10 14:42:41.012116 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:41.012082    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:41.012655 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.012248    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:41.181555 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:41.181410    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal" event={"ID":"d2d6563cf27d06d8ea6209c65b6252bc","Type":"ContainerStarted","Data":"49f01c7ee3aad59ca770dc0d40aafa428f6017aa518a39d5d4cdd583484b11e8"}
Sep 10 14:42:41.521953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:41.521913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:41.522169 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.522135    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:41.522270 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.522205    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs podName:0fb75b7e-b7c0-4bfa-8c51-9b96e4449391 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:45.522184515 +0000 UTC m=+10.008872401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs") pod "network-metrics-daemon-hbxnv" (UID: "0fb75b7e-b7c0-4bfa-8c51-9b96e4449391") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:41.623692 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:41.623625    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:41.623879 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.623802    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:41.623879 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.623868    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret podName:c1938126-5194-491b-b8e3-a602e268c691 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:45.623851274 +0000 UTC m=+10.110539144 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret") pod "global-pull-secret-syncer-g7dg8" (UID: "c1938126-5194-491b-b8e3-a602e268c691") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:41.724369 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:41.724325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:41.724552 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.724504    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:42:41.724552 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.724529    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:42:41.724552 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.724543    2558 projected.go:196] Error preparing data for projected volume kube-api-access-s6hpv for pod openshift-network-diagnostics/network-check-target-jt5pz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:41.724749 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:41.724604    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv podName:f0076e40-95a9-473a-bfdc-2e93c30e9f7d nodeName:}" failed. No retries permitted until 2026-09-10 14:42:45.724583405 +0000 UTC m=+10.211271277 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s6hpv" (UniqueName: "kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv") pod "network-check-target-jt5pz" (UID: "f0076e40-95a9-473a-bfdc-2e93c30e9f7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:42.012493 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:42.012400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:42.012997 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:42.012559    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:42.012997 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:42.012956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:42.013127 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:42.013051    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:43.013024 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:43.012969    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:43.013649 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:43.013109    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:44.012734 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:44.012695    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:44.012914 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:44.012853    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:44.013577 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:44.013408    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:44.013577 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:44.013526    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:45.012626 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:45.012554    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:45.012766 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.012693    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:45.565184 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:45.565144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:45.565675 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.565350    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:45.565675 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.565418    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs podName:0fb75b7e-b7c0-4bfa-8c51-9b96e4449391 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:53.565397073 +0000 UTC m=+18.052084946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs") pod "network-metrics-daemon-hbxnv" (UID: "0fb75b7e-b7c0-4bfa-8c51-9b96e4449391") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:45.665827 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:45.665786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:45.666013 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.665966    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:45.666075 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.666039    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret podName:c1938126-5194-491b-b8e3-a602e268c691 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:53.666019614 +0000 UTC m=+18.152707489 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret") pod "global-pull-secret-syncer-g7dg8" (UID: "c1938126-5194-491b-b8e3-a602e268c691") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:45.767087 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:45.766530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:45.767087 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.766679    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:42:45.767087 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.766696    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:42:45.767087 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.766706    2558 projected.go:196] Error preparing data for projected volume kube-api-access-s6hpv for pod openshift-network-diagnostics/network-check-target-jt5pz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:45.767087 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:45.766761    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv podName:f0076e40-95a9-473a-bfdc-2e93c30e9f7d nodeName:}" failed. No retries permitted until 2026-09-10 14:42:53.766742187 +0000 UTC m=+18.253430076 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s6hpv" (UniqueName: "kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv") pod "network-check-target-jt5pz" (UID: "f0076e40-95a9-473a-bfdc-2e93c30e9f7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:46.015873 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:46.015793    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:46.016040 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:46.015926    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:46.016329 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:46.016308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:46.016447 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:46.016405    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:47.012098 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:47.012066    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:47.012660 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:47.012183    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:48.012605 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:48.012488    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:48.013058 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:48.012636    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:48.013058 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:48.013009    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:48.013174 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:48.013118    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:49.012091 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:49.012059    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:49.012254 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:49.012194    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:50.012508 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:50.012477    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:50.012838 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:50.012580    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:50.012838 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:50.012621    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:50.012838 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:50.012675    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:51.013042 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:51.013009    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:51.013462 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:51.013136    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:52.012689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:52.012657    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:52.012847 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:52.012658    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:52.012847 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:52.012823    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:52.012978 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:52.012950    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:53.012481 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:53.012453    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:53.012919 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.012576    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:53.627372 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:53.627338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:53.627626 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.627519    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:53.627626 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.627596    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs podName:0fb75b7e-b7c0-4bfa-8c51-9b96e4449391 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:09.627573925 +0000 UTC m=+34.114261797 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs") pod "network-metrics-daemon-hbxnv" (UID: "0fb75b7e-b7c0-4bfa-8c51-9b96e4449391") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:42:53.727990 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:53.727953    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:53.728159 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.728071    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:53.728159 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.728133    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret podName:c1938126-5194-491b-b8e3-a602e268c691 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:09.728117099 +0000 UTC m=+34.214804971 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret") pod "global-pull-secret-syncer-g7dg8" (UID: "c1938126-5194-491b-b8e3-a602e268c691") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:42:53.828764 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:53.828732    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:53.828916 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.828868    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:42:53.828916 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.828882    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:42:53.828916 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.828891    2558 projected.go:196] Error preparing data for projected volume kube-api-access-s6hpv for pod openshift-network-diagnostics/network-check-target-jt5pz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:53.829060 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:53.828950    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv podName:f0076e40-95a9-473a-bfdc-2e93c30e9f7d nodeName:}" failed. No retries permitted until 2026-09-10 14:43:09.82893381 +0000 UTC m=+34.315621697 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s6hpv" (UniqueName: "kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv") pod "network-check-target-jt5pz" (UID: "f0076e40-95a9-473a-bfdc-2e93c30e9f7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:42:54.012689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:54.012659    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:54.013053 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:54.012665    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:54.013053 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:54.012776    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:54.013053 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:54.012829    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:55.013090 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:55.013051    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:55.013559 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:55.013193    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:56.013589 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:56.013556    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:56.014018 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:56.013651    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:56.014018 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:56.013751    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:56.014018 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:56.013868    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:57.012863 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:57.012817    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:57.013039 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:57.012961    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:58.012325 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.012015    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:42:58.013101 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.012076    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:42:58.013101 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:58.012414    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:42:58.013101 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:58.012466    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:42:58.214604 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.214577    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vp8pc" event={"ID":"de043964-4505-4d6c-9af0-62ba8b338a9d","Type":"ContainerStarted","Data":"bb4262aae99bd07f4597e875dae593d1030196597885ac317c7bafbb47294441"}
Sep 10 14:42:58.215924 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.215903    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nmjz6" event={"ID":"11d4c95f-8587-43ee-addb-325df5853d41","Type":"ContainerStarted","Data":"b8eff9b9cec04deef5344d06e220c304e8017a594961545c008bf473d25591bf"}
Sep 10 14:42:58.217254 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.217234    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qglhm" event={"ID":"523956a8-c95c-44d7-8a58-48997ea2f395","Type":"ContainerStarted","Data":"1b0d80f53aedd1ef6b105fd3b3ceefb2bddb9084467a20b5438a91c381e1121c"}
Sep 10 14:42:58.218646 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.218629    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zfx7h" event={"ID":"d36866a7-ce48-4c5f-b14a-b3cad558f061","Type":"ContainerStarted","Data":"705a5d732b53212ddca9ae91aff02a293e3b4dfabb26991ab4d1be61f1e1f4e9"}
Sep 10 14:42:58.223817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.223783    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"d8c01278e5d3d769fa51e42c44b67b8f909c9c1d57f00d24678932bf32846d18"}
Sep 10 14:42:58.223912 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.223887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"c5bf55bd01ec87468bddf8f6648178343059545c755cbad64e6eae4abc939f06"}
Sep 10 14:42:58.223984 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.223919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"d74fca993f591e5bee29522086ed758d40bc3003f7fe63eebd97b8b25ef01075"}
Sep 10 14:42:58.223984 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.223947    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"4e5f67897fb4b63e4687eee678646e821b6fdae2b1a8c47f8773e770c2e6a234"}
Sep 10 14:42:58.224059 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.223990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"467b68eacdd15915fe525bb4c61e9d5ea848a25abd014a25e6f848cc8dcec28a"}
Sep 10 14:42:58.225107 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.225087    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv" event={"ID":"71c85297-913a-4b34-86e4-83f1acb6b631","Type":"ContainerStarted","Data":"ad5aa958ade1563ac94677238878dc47fed02e6f3212f56b6a126a685d616e65"}
Sep 10 14:42:58.226301 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.226280    2558 generic.go:356] "Generic (PLEG): container finished" podID="0a44e73c-8b44-4d88-97b0-ce0446954353" containerID="2773279eb6fee64de0bb4aaf52fdbb8cb0c3ec138d098446cb3612c9c1f0c58a" exitCode=0
Sep 10 14:42:58.226375 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.226350    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerDied","Data":"2773279eb6fee64de0bb4aaf52fdbb8cb0c3ec138d098446cb3612c9c1f0c58a"}
Sep 10 14:42:58.227686 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.227654    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8bp45" event={"ID":"cbb51f27-bb88-427d-8e1b-86bd53b34d73","Type":"ContainerStarted","Data":"143420e8609e31d38e4c24bc79ca3983c59190494b71191d3296914e499f89f7"}
Sep 10 14:42:58.247702 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.247655    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-253.ec2.internal" podStartSLOduration=21.247635873 podStartE2EDuration="21.247635873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:41.198620972 +0000 UTC m=+5.685308867" watchObservedRunningTime="2026-09-10 14:42:58.247635873 +0000 UTC m=+22.734323765"
Sep 10 14:42:58.263198 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.263152    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vp8pc" podStartSLOduration=7.205255797 podStartE2EDuration="22.263136299s" totalImagesPullingTime="15.057880502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:58.262904967 +0000 UTC m=+22.749592857" watchObservedRunningTime="2026-09-10 14:42:58.263136299 +0000 UTC m=+22.749824190"
Sep 10 14:42:58.278699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.278660    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nmjz6" podStartSLOduration=3.87068565 podStartE2EDuration="22.278643903s" totalImagesPullingTime="18.407958253s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:58.278060821 +0000 UTC m=+22.764748711" watchObservedRunningTime="2026-09-10 14:42:58.278643903 +0000 UTC m=+22.765331794"
Sep 10 14:42:58.310538 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.310484    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8bp45" podStartSLOduration=3.878175755 podStartE2EDuration="22.310462503s" totalImagesPullingTime="18.432286748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:58.294660075 +0000 UTC m=+22.781347965" watchObservedRunningTime="2026-09-10 14:42:58.310462503 +0000 UTC m=+22.797150396"
Sep 10 14:42:58.310665 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.310630    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qglhm" podStartSLOduration=3.902309817 podStartE2EDuration="22.310618907s" totalImagesPullingTime="18.40830909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:58.309587184 +0000 UTC m=+22.796275087" watchObservedRunningTime="2026-09-10 14:42:58.310618907 +0000 UTC m=+22.797306816"
Sep 10 14:42:58.327286 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:58.327210    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zfx7h" podStartSLOduration=3.865515709 podStartE2EDuration="22.327193225s" totalImagesPullingTime="18.461677516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:58.326765722 +0000 UTC m=+22.813453613" watchObservedRunningTime="2026-09-10 14:42:58.327193225 +0000 UTC m=+22.813881136"
Sep 10 14:42:59.012778 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:59.012752    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:42:59.013163 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:42:59.012855    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:42:59.231462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:59.231425    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2tvr6" event={"ID":"bcfb93f7-db68-46f7-8445-b9c768f63b8a","Type":"ContainerStarted","Data":"fc1753d755026d7079aa23923ec4834e388fcdad773605ca4c29e2e963e91596"}
Sep 10 14:42:59.236106 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:59.235885    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"05bf73987f4b98cdf669f3c25813b9de1a3ab70cbe4a0454042e3da6fb2358e5"}
Sep 10 14:42:59.255386 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:59.255364    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 14:42:59.949859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:59.949750    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T14:42:59.255379878Z","UUID":"08b2ead6-bbd7-4163-bb66-d873a33c7c19","Handler":null,"Name":"","Endpoint":""}
Sep 10 14:42:59.951564 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:59.951538    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 14:42:59.951564 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:42:59.951570    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 14:43:00.013305 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:00.013264    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:00.013848 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:00.013308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:00.013848 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:00.013410    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:00.013848 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:00.013818    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:00.239985 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:00.239897    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv" event={"ID":"71c85297-913a-4b34-86e4-83f1acb6b631","Type":"ContainerStarted","Data":"ea8b81c21ccb0f37acd6b652c59736c3b55f64d45b0a347be089ceef70c4194d"}
Sep 10 14:43:01.012179 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:01.012103    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:01.012338 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:01.012204    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:01.245808 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:01.245771    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"b4073402b820570035baa95abe73434992bd99506d98dc253238a03a71c3019f"}
Sep 10 14:43:01.248081 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:01.248048    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv" event={"ID":"71c85297-913a-4b34-86e4-83f1acb6b631","Type":"ContainerStarted","Data":"1a19def69c535332b0138ddda61e3d37351aac36ed43b9ab5d4fe59d65107921"}
Sep 10 14:43:01.266926 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:01.266849    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4bqsv" podStartSLOduration=4.400043732 podStartE2EDuration="25.266832582s" totalImagesPullingTime="20.86678885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:01.266808641 +0000 UTC m=+25.753496534" watchObservedRunningTime="2026-09-10 14:43:01.266832582 +0000 UTC m=+25.753520454"
Sep 10 14:43:01.267359 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:01.267323    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2tvr6" podStartSLOduration=6.832911366 podStartE2EDuration="25.267307106s" totalImagesPullingTime="18.43439574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:59.246940407 +0000 UTC m=+23.733628297" watchObservedRunningTime="2026-09-10 14:43:01.267307106 +0000 UTC m=+25.753994998"
Sep 10 14:43:01.465131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:01.465095    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:43:01.465961 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:01.465939    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:43:02.012456 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:02.012424    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:02.012456 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:02.012462    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:02.012695 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:02.012564    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:02.012752 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:02.012711    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:03.012642 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:03.012461    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:03.013297 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:03.012728    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:03.254805 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:03.254773    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerStarted","Data":"2d3abde31cf5a9478657dcca94a3eb487fc0b40e564c93ed65305d6fc49b3548"}
Sep 10 14:43:03.258659 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:03.258634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" event={"ID":"21e67658-4290-47db-ae99-95df1ceebb2d","Type":"ContainerStarted","Data":"873c859edca3ce5f70749d65de656d38883dba0b1f4912765d7d998efbb5f692"}
Sep 10 14:43:03.258894 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:03.258873    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:43:03.258980 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:03.258932    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:43:03.274580 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:03.274556    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:43:03.335188 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:03.335087    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt" podStartSLOduration=8.829559723 podStartE2EDuration="27.335071516s" totalImagesPullingTime="18.505511793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:03.334874213 +0000 UTC m=+27.821562117" watchObservedRunningTime="2026-09-10 14:43:03.335071516 +0000 UTC m=+27.821759407"
Sep 10 14:43:04.013251 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.012975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:04.013970 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:04.013349    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:04.013970 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.013805    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:04.013970 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:04.013917    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:04.262938 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.262901    2558 generic.go:356] "Generic (PLEG): container finished" podID="0a44e73c-8b44-4d88-97b0-ce0446954353" containerID="2d3abde31cf5a9478657dcca94a3eb487fc0b40e564c93ed65305d6fc49b3548" exitCode=0
Sep 10 14:43:04.263163 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.262991    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerDied","Data":"2d3abde31cf5a9478657dcca94a3eb487fc0b40e564c93ed65305d6fc49b3548"}
Sep 10 14:43:04.263643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.263559    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:43:04.279814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.279785    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:43:04.702592 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.702561    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g7dg8"]
Sep 10 14:43:04.702808 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.702792    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:04.702940 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:04.702912    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:04.705555 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.705533    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hbxnv"]
Sep 10 14:43:04.705654 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.705642    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:04.705781 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:04.705760    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:04.720462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.720437    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jt5pz"]
Sep 10 14:43:04.720575 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.720529    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:04.720619 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:04.720602    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:04.837074 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:04.837012    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qglhm_523956a8-c95c-44d7-8a58-48997ea2f395/dns-node-resolver/0.log"
Sep 10 14:43:05.806394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:05.806365    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nmjz6_11d4c95f-8587-43ee-addb-325df5853d41/node-ca/0.log"
Sep 10 14:43:06.013714 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:06.013636    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:06.013856 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:06.013728    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:06.013856 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:06.013810    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:06.013924 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:06.013899    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:06.013962 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:06.013936    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:06.013994 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:06.013986    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:06.270304 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:06.270205    2558 generic.go:356] "Generic (PLEG): container finished" podID="0a44e73c-8b44-4d88-97b0-ce0446954353" containerID="55f60958502496a813fbbe5325e7a3683b0e11bfbad7e81ea3bee52bf8d319cf" exitCode=0
Sep 10 14:43:06.270304 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:06.270291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerDied","Data":"55f60958502496a813fbbe5325e7a3683b0e11bfbad7e81ea3bee52bf8d319cf"}
Sep 10 14:43:07.924009 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:07.923824    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:43:07.924450 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:07.924159    2558 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 14:43:07.924497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:07.924442    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vp8pc"
Sep 10 14:43:08.013079 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:08.013047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:08.013247 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:08.013047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:08.013247 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:08.013154    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:08.013339 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:08.013264    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:08.013339 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:08.013050    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:08.013398 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:08.013360    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:08.277512 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:08.277434    2558 generic.go:356] "Generic (PLEG): container finished" podID="0a44e73c-8b44-4d88-97b0-ce0446954353" containerID="bc27e1c989aab1eb347befdf6adfce57814bb5132e06f3c8413b72edfd7dce09" exitCode=0
Sep 10 14:43:08.277645 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:08.277520    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerDied","Data":"bc27e1c989aab1eb347befdf6adfce57814bb5132e06f3c8413b72edfd7dce09"}
Sep 10 14:43:09.644394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:09.644353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:09.644968 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.644537    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:43:09.644968 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.644601    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs podName:0fb75b7e-b7c0-4bfa-8c51-9b96e4449391 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:41.644585122 +0000 UTC m=+66.131272991 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs") pod "network-metrics-daemon-hbxnv" (UID: "0fb75b7e-b7c0-4bfa-8c51-9b96e4449391") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:43:09.745578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:09.745545    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:09.745754 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.745665    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:43:09.745754 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.745723    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret podName:c1938126-5194-491b-b8e3-a602e268c691 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:41.74570405 +0000 UTC m=+66.232391919 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret") pod "global-pull-secret-syncer-g7dg8" (UID: "c1938126-5194-491b-b8e3-a602e268c691") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:43:09.846817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:09.846770    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:09.847008 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.846943    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:43:09.847008 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.846969    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:43:09.847008 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.846980    2558 projected.go:196] Error preparing data for projected volume kube-api-access-s6hpv for pod openshift-network-diagnostics/network-check-target-jt5pz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:43:09.847173 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:09.847030    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv podName:f0076e40-95a9-473a-bfdc-2e93c30e9f7d nodeName:}" failed. No retries permitted until 2026-09-10 14:43:41.84701628 +0000 UTC m=+66.333704148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s6hpv" (UniqueName: "kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv") pod "network-check-target-jt5pz" (UID: "f0076e40-95a9-473a-bfdc-2e93c30e9f7d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:43:10.012935 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:10.012854    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:10.012935 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:10.012910    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:10.012935 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:10.012924    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:10.013159 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:10.013002    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:10.013159 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:10.013073    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:10.013260 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:10.013157    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:12.012639 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:12.012605    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:12.013158 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:12.012604    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:12.013158 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:12.012750    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:12.013158 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:12.012605    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:12.013158 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:12.012800    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:12.013158 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:12.012871    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:13.292901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:13.292816    2558 generic.go:356] "Generic (PLEG): container finished" podID="0a44e73c-8b44-4d88-97b0-ce0446954353" containerID="aa208b69fd333e4a7db339013648aeb29fe087d6064ed483acded31f8bb1a9bd" exitCode=0
Sep 10 14:43:13.292901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:13.292876    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerDied","Data":"aa208b69fd333e4a7db339013648aeb29fe087d6064ed483acded31f8bb1a9bd"}
Sep 10 14:43:14.012831 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:14.012792    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:14.013005 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:14.012800    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:14.013005 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:14.012894    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:14.013005 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:14.012913    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:14.013133 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:14.013004    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:14.013133 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:14.013083    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:14.297757 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:14.297715    2558 generic.go:356] "Generic (PLEG): container finished" podID="0a44e73c-8b44-4d88-97b0-ce0446954353" containerID="dcfad6ec7b10ddf041891e000ddbfb41c51f3b5f5bb69910e52dde5e4d746bcf" exitCode=0
Sep 10 14:43:14.298339 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:14.297776    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerDied","Data":"dcfad6ec7b10ddf041891e000ddbfb41c51f3b5f5bb69910e52dde5e4d746bcf"}
Sep 10 14:43:15.303813 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:15.303778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8nc57" event={"ID":"0a44e73c-8b44-4d88-97b0-ce0446954353","Type":"ContainerStarted","Data":"352379ea4d916311062773647dd0e9e5aa4eacc83896ce72d4258b2312e0c3f6"}
Sep 10 14:43:15.325269 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:15.325192    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8nc57" podStartSLOduration=9.040601696 podStartE2EDuration="39.325168377s" totalImagesPullingTime="30.284566681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:15.325042367 +0000 UTC m=+39.811730260" watchObservedRunningTime="2026-09-10 14:43:15.325168377 +0000 UTC m=+39.811856268"
Sep 10 14:43:16.012883 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:16.012849    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:16.013039 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:16.012933    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:16.013039 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:16.012992    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:16.013109 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:16.013079    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:16.013109 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:16.013099    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:16.013189 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:16.013173    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:18.012309 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:18.012279    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:18.012309 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:18.012306    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:18.012706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:18.012279    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:18.012706 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:18.012373    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:18.012706 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:18.012466    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:18.012706 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:18.012526    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:20.012560 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:20.012525    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:20.012905 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:20.012534    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:20.012905 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:20.012657    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:20.012905 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:20.012657    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:20.012905 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:20.012738    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:20.012905 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:20.012817    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:22.013002 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:22.012963    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:22.013465 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:22.012963    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:22.013465 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:22.013077    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:22.013465 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:22.013175    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:22.013465 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:22.012963    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:22.013465 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:22.013297    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:24.012528 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:24.012500    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:24.012897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:24.012498    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:24.012897 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:24.012597    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:24.012897 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:24.012691    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:24.012897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:24.012740    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:24.012897 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:24.012823    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:26.013354 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.013318    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:26.013830 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.013421    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:26.013830 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:26.013476    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbxnv" podUID="0fb75b7e-b7c0-4bfa-8c51-9b96e4449391"
Sep 10 14:43:26.013830 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.013508    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:26.013830 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:26.013573    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7dg8" podUID="c1938126-5194-491b-b8e3-a602e268c691"
Sep 10 14:43:26.013830 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:26.013493    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jt5pz" podUID="f0076e40-95a9-473a-bfdc-2e93c30e9f7d"
Sep 10 14:43:26.237610 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.237581    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-253.ec2.internal" event="NodeReady"
Sep 10 14:43:26.237799 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.237731    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 14:43:26.292902 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.292813    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bx5ht"]
Sep 10 14:43:26.303851 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.303825    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wh2cr"]
Sep 10 14:43:26.304007 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.303970    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.306531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.306506    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mkqg5\""
Sep 10 14:43:26.306644 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.306535    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 14:43:26.306644 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.306506    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 14:43:26.315263 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.315237    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bx5ht"]
Sep 10 14:43:26.315364 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.315278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wh2cr"]
Sep 10 14:43:26.315364 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.315338    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.317687 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.317668    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 14:43:26.317807 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.317779    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 14:43:26.317807 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.317786    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 14:43:26.317904 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.317823    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 14:43:26.317960 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.317906    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fmx4m\""
Sep 10 14:43:26.390659 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.390626    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j82z2"]
Sep 10 14:43:26.406214 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.406183    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j82z2"]
Sep 10 14:43:26.406392 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.406326    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.408861 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.408840    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 14:43:26.409018 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.408998    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 14:43:26.409094 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.409028    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 14:43:26.409312 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.409293    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s4dwj\""
Sep 10 14:43:26.459712 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459670    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.459712 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-crio-socket\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.459909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459736    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80829f4e-d9fa-4a86-9d34-371483048e09-tmp-dir\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.459909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459797    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80829f4e-d9fa-4a86-9d34-371483048e09-metrics-tls\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.459909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459830    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr4ds\" (UniqueName: \"kubernetes.io/projected/80829f4e-d9fa-4a86-9d34-371483048e09-kube-api-access-jr4ds\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.459909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-data-volume\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.459909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459885    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.460053 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459940    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9cx\" (UniqueName: \"kubernetes.io/projected/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-kube-api-access-kz9cx\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.460053 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.459969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80829f4e-d9fa-4a86-9d34-371483048e09-config-volume\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.560357 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560278    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w27xb\" (UniqueName: \"kubernetes.io/projected/43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d-kube-api-access-w27xb\") pod \"ingress-canary-j82z2\" (UID: \"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d\") " pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.560357 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560315    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.560357 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-crio-socket\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.560357 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80829f4e-d9fa-4a86-9d34-371483048e09-tmp-dir\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560409    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80829f4e-d9fa-4a86-9d34-371483048e09-metrics-tls\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560440    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr4ds\" (UniqueName: \"kubernetes.io/projected/80829f4e-d9fa-4a86-9d34-371483048e09-kube-api-access-jr4ds\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d-cert\") pod \"ingress-canary-j82z2\" (UID: \"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d\") " pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-data-volume\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-crio-socket\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560568    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9cx\" (UniqueName: \"kubernetes.io/projected/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-kube-api-access-kz9cx\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.560633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560614    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80829f4e-d9fa-4a86-9d34-371483048e09-config-volume\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.560940 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560734    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80829f4e-d9fa-4a86-9d34-371483048e09-tmp-dir\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.560940 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-data-volume\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.561005 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.560985    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.561104 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.561087    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/80829f4e-d9fa-4a86-9d34-371483048e09-config-volume\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.564472 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.564449    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.564584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.564455    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/80829f4e-d9fa-4a86-9d34-371483048e09-metrics-tls\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.570606 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.570583    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9cx\" (UniqueName: \"kubernetes.io/projected/eb1b0606-9efe-4e97-8a0e-b06d4eecf18b-kube-api-access-kz9cx\") pod \"insights-runtime-extractor-wh2cr\" (UID: \"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b\") " pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.571376 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.571360    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr4ds\" (UniqueName: \"kubernetes.io/projected/80829f4e-d9fa-4a86-9d34-371483048e09-kube-api-access-jr4ds\") pod \"dns-default-bx5ht\" (UID: \"80829f4e-d9fa-4a86-9d34-371483048e09\") " pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.614767 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.614732    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:26.623909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.623884    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wh2cr"
Sep 10 14:43:26.661818 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.661786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d-cert\") pod \"ingress-canary-j82z2\" (UID: \"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d\") " pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.661948 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.661885    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w27xb\" (UniqueName: \"kubernetes.io/projected/43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d-kube-api-access-w27xb\") pod \"ingress-canary-j82z2\" (UID: \"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d\") " pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.664395 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.664369    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d-cert\") pod \"ingress-canary-j82z2\" (UID: \"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d\") " pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.670463 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.670437    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w27xb\" (UniqueName: \"kubernetes.io/projected/43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d-kube-api-access-w27xb\") pod \"ingress-canary-j82z2\" (UID: \"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d\") " pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.714411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.714382    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j82z2"
Sep 10 14:43:26.931880 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.931809    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bx5ht"]
Sep 10 14:43:26.934875 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:26.934851    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wh2cr"]
Sep 10 14:43:26.935966 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:26.935945    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80829f4e_d9fa_4a86_9d34_371483048e09.slice/crio-2e2bef9a986893ec1822088132682f7932780244e49f797b9fcc29982241154f WatchSource:0}: Error finding container 2e2bef9a986893ec1822088132682f7932780244e49f797b9fcc29982241154f: Status 404 returned error can't find the container with id 2e2bef9a986893ec1822088132682f7932780244e49f797b9fcc29982241154f
Sep 10 14:43:26.937914 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:26.937891    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1b0606_9efe_4e97_8a0e_b06d4eecf18b.slice/crio-113c34babce0f7431d90ef90513f4a0977a25adf025fcb44b7b77f54d0db1b06 WatchSource:0}: Error finding container 113c34babce0f7431d90ef90513f4a0977a25adf025fcb44b7b77f54d0db1b06: Status 404 returned error can't find the container with id 113c34babce0f7431d90ef90513f4a0977a25adf025fcb44b7b77f54d0db1b06
Sep 10 14:43:27.001998 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.001968    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j82z2"]
Sep 10 14:43:27.005652 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:27.005623    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d5b4a5_d9d8_4a8b_bda9_0cb39deaab6d.slice/crio-99ab07ebc5d2c67f1159f68a9fe83097befa1142d477004cfa81f27f07a17fdc WatchSource:0}: Error finding container 99ab07ebc5d2c67f1159f68a9fe83097befa1142d477004cfa81f27f07a17fdc: Status 404 returned error can't find the container with id 99ab07ebc5d2c67f1159f68a9fe83097befa1142d477004cfa81f27f07a17fdc
Sep 10 14:43:27.330785 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.330747    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j82z2" event={"ID":"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d","Type":"ContainerStarted","Data":"99ab07ebc5d2c67f1159f68a9fe83097befa1142d477004cfa81f27f07a17fdc"}
Sep 10 14:43:27.332156 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.332132    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wh2cr" event={"ID":"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b","Type":"ContainerStarted","Data":"ce5548d5640bb800ac9ccfcf7650ecd15b7ff33829c1c316717c041e1ef80d6b"}
Sep 10 14:43:27.332292 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.332186    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wh2cr" event={"ID":"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b","Type":"ContainerStarted","Data":"113c34babce0f7431d90ef90513f4a0977a25adf025fcb44b7b77f54d0db1b06"}
Sep 10 14:43:27.333157 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.333138    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bx5ht" event={"ID":"80829f4e-d9fa-4a86-9d34-371483048e09","Type":"ContainerStarted","Data":"2e2bef9a986893ec1822088132682f7932780244e49f797b9fcc29982241154f"}
Sep 10 14:43:27.494480 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.494446    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"]
Sep 10 14:43:27.510968 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.510930    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"]
Sep 10 14:43:27.511185 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.511069    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.514412 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.514353    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 14:43:27.514543 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.514448    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bfbcc\""
Sep 10 14:43:27.514543 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.514518    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 14:43:27.515156 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.515136    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 14:43:27.515272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.515162    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 14:43:27.515272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.515172    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 14:43:27.670404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.670319    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.670404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.670353    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173b337b-ac39-405d-aab9-06d3b35fb3a7-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.670404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.670391    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52592\" (UniqueName: \"kubernetes.io/projected/173b337b-ac39-405d-aab9-06d3b35fb3a7-kube-api-access-52592\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.670685 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.670417    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.771608 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.771566    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.771757 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.771615    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173b337b-ac39-405d-aab9-06d3b35fb3a7-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.771757 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.771711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52592\" (UniqueName: \"kubernetes.io/projected/173b337b-ac39-405d-aab9-06d3b35fb3a7-kube-api-access-52592\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.771853 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.771755    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.771921 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:27.771903    2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 10 14:43:27.771976 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:27.771968    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-tls podName:173b337b-ac39-405d-aab9-06d3b35fb3a7 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:28.271949677 +0000 UTC m=+52.758637565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-zmfb2" (UID: "173b337b-ac39-405d-aab9-06d3b35fb3a7") : secret "prometheus-operator-tls" not found
Sep 10 14:43:27.772576 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.772543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173b337b-ac39-405d-aab9-06d3b35fb3a7-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.774778 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.774753    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:27.782469 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:27.782444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52592\" (UniqueName: \"kubernetes.io/projected/173b337b-ac39-405d-aab9-06d3b35fb3a7-kube-api-access-52592\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:28.012566 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.012531    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:28.012783 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.012531    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:28.013085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.012531    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:28.015421 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.015399    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:43:28.015522 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.015399    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:43:28.016429 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.016406    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c8srj\""
Sep 10 14:43:28.016647 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.016620    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:43:28.016861 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.016845    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:43:28.016943 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.016888    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnf6m\""
Sep 10 14:43:28.276134 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.276052    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:28.278984 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.278931    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/173b337b-ac39-405d-aab9-06d3b35fb3a7-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-zmfb2\" (UID: \"173b337b-ac39-405d-aab9-06d3b35fb3a7\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:28.339133 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.339093    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wh2cr" event={"ID":"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b","Type":"ContainerStarted","Data":"4d2fd1d88db7b4cc3f245ab7f61c6ea5c6d52d6976a514e014196b62a622927f"}
Sep 10 14:43:28.422364 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:28.422333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"
Sep 10 14:43:29.050490 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:29.050318    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-zmfb2"]
Sep 10 14:43:29.343944 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:29.343850    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bx5ht" event={"ID":"80829f4e-d9fa-4a86-9d34-371483048e09","Type":"ContainerStarted","Data":"9b4e59e0e96bb50e0349a770281075bef47a3c540e22ad312939f23ef0098d5a"}
Sep 10 14:43:29.762059 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:29.762029    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173b337b_ac39_405d_aab9_06d3b35fb3a7.slice/crio-72e92fdf903dcd1c1a3a0656b178a78ecb8ad5d1c97eb82b358118c0c29d3590 WatchSource:0}: Error finding container 72e92fdf903dcd1c1a3a0656b178a78ecb8ad5d1c97eb82b358118c0c29d3590: Status 404 returned error can't find the container with id 72e92fdf903dcd1c1a3a0656b178a78ecb8ad5d1c97eb82b358118c0c29d3590
Sep 10 14:43:30.350540 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.350498    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wh2cr" event={"ID":"eb1b0606-9efe-4e97-8a0e-b06d4eecf18b","Type":"ContainerStarted","Data":"071e2eeff7aed8d3457c66e18099916d414d0dd84e6f93bc1d9d448e47c32e43"}
Sep 10 14:43:30.352237 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.352186    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2" event={"ID":"173b337b-ac39-405d-aab9-06d3b35fb3a7","Type":"ContainerStarted","Data":"72e92fdf903dcd1c1a3a0656b178a78ecb8ad5d1c97eb82b358118c0c29d3590"}
Sep 10 14:43:30.353890 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.353866    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j82z2" event={"ID":"43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d","Type":"ContainerStarted","Data":"08cae19dbbd8d9acea8a883b27e989d638669fcba241d9bd0643cd53706d6ba0"}
Sep 10 14:43:30.356210 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.356186    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bx5ht" event={"ID":"80829f4e-d9fa-4a86-9d34-371483048e09","Type":"ContainerStarted","Data":"b72d30410e0b144159ad0bb27cea47fcd3bc7a20ee50692ced85124beae26020"}
Sep 10 14:43:30.356360 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.356339    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:30.371187 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.371132    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wh2cr" podStartSLOduration=1.402762381 podStartE2EDuration="4.371113623s" totalImagesPullingTime="2.968351242s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:43:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:30.37032573 +0000 UTC m=+54.857013650" watchObservedRunningTime="2026-09-10 14:43:30.371113623 +0000 UTC m=+54.857801517"
Sep 10 14:43:30.432924 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.432804    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bx5ht" podStartSLOduration=2.441204174 podStartE2EDuration="4.432780486s" totalImagesPullingTime="1.991576312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:30.404957375 +0000 UTC m=+54.891645266" watchObservedRunningTime="2026-09-10 14:43:30.432780486 +0000 UTC m=+54.919468378"
Sep 10 14:43:30.433091 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:30.432934    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j82z2" podStartSLOduration=1.282612807 podStartE2EDuration="4.432926637s" totalImagesPullingTime="3.15031383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:30.432442465 +0000 UTC m=+54.919130356" watchObservedRunningTime="2026-09-10 14:43:30.432926637 +0000 UTC m=+54.919614529"
Sep 10 14:43:31.362567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:31.362532    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2" event={"ID":"173b337b-ac39-405d-aab9-06d3b35fb3a7","Type":"ContainerStarted","Data":"76f4922781953aee8da70afb4024230a36b4d2e541474ce9bf67d04d658f880e"}
Sep 10 14:43:31.362922 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:31.362589    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2" event={"ID":"173b337b-ac39-405d-aab9-06d3b35fb3a7","Type":"ContainerStarted","Data":"0359044b3bf96ffd501945b3340bb92349871cd3de803ac336a6e3d1a4b652fb"}
Sep 10 14:43:31.384082 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:31.384030    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-zmfb2" podStartSLOduration=3.234704083 podStartE2EDuration="4.384013863s" totalImagesPullingTime="1.14930978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:31.38275095 +0000 UTC m=+55.869438840" watchObservedRunningTime="2026-09-10 14:43:31.384013863 +0000 UTC m=+55.870701754"
Sep 10 14:43:32.857141 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.857106    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"]
Sep 10 14:43:32.859150 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.859134    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:32.861424 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.861401    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 14:43:32.861573 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.861503    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2jv99\""
Sep 10 14:43:32.861662 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.861644    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 14:43:32.872185 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.872160    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"]
Sep 10 14:43:32.872859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.872838    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"]
Sep 10 14:43:32.874602 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.874587    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:32.877154 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.877133    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 14:43:32.877307 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.877194    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 14:43:32.877689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.877672    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2nj8b\""
Sep 10 14:43:32.877920 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.877898    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 14:43:32.887536 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.887515    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"]
Sep 10 14:43:32.893586 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.893568    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-89hhw"]
Sep 10 14:43:32.895497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.895483    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.898198 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.898181    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 14:43:32.898397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.898366    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 14:43:32.898519 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.898403    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 14:43:32.898519 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.898485    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vfxxz\""
Sep 10 14:43:32.909552 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xh5z\" (UniqueName: \"kubernetes.io/projected/35527307-227f-4f31-a7e0-fa2c7fa7e13d-kube-api-access-4xh5z\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:32.909631 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35527307-227f-4f31-a7e0-fa2c7fa7e13d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:32.909631 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909577    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:32.909631 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909603    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:32.909766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909641    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ngm6\" (UniqueName: \"kubernetes.io/projected/ca37333a-5822-44cd-8482-e673c1cb7d18-kube-api-access-4ngm6\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.909766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909657    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/35527307-227f-4f31-a7e0-fa2c7fa7e13d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:32.909766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909673    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35527307-227f-4f31-a7e0-fa2c7fa7e13d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:32.909766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:32.909766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f6sg\" (UniqueName: \"kubernetes.io/projected/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-api-access-7f6sg\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:32.909766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909765    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-wtmp\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.909952 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909828    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-root\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.909952 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909851    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-tls\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.909952 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909866    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca37333a-5822-44cd-8482-e673c1cb7d18-metrics-client-ca\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.909952 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909882    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34467f13-07d0-4d86-bdec-caa7c2098e84-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:32.910137 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-textfile\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.910137 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.909986    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.910137 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.910016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-accelerators-collector-config\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:32.910137 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.910044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/34467f13-07d0-4d86-bdec-caa7c2098e84-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:32.910137 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:32.910071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-sys\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011245 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011186    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/35527307-227f-4f31-a7e0-fa2c7fa7e13d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.011394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011259    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35527307-227f-4f31-a7e0-fa2c7fa7e13d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.011394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011310    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.011394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f6sg\" (UniqueName: \"kubernetes.io/projected/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-api-access-7f6sg\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.011394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011370    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-wtmp\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011593 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:33.011404    2558 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 10 14:43:33.011593 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:33.011476    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-tls podName:34467f13-07d0-4d86-bdec-caa7c2098e84 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:33.511455886 +0000 UTC m=+57.998143778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-k4nzs" (UID: "34467f13-07d0-4d86-bdec-caa7c2098e84") : secret "kube-state-metrics-tls" not found
Sep 10 14:43:33.011593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011528    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-wtmp\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011737 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011622    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-root\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011737 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011650    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-tls\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011737 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011679    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca37333a-5822-44cd-8482-e673c1cb7d18-metrics-client-ca\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011737 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011728    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34467f13-07d0-4d86-bdec-caa7c2098e84-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.011966 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011757    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-textfile\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011966 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-root\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011966 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:33.011767    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 14:43:33.011966 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011966 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:33.011840    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-tls podName:ca37333a-5822-44cd-8482-e673c1cb7d18 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:33.511820549 +0000 UTC m=+57.998508434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-tls") pod "node-exporter-89hhw" (UID: "ca37333a-5822-44cd-8482-e673c1cb7d18") : secret "node-exporter-tls" not found
Sep 10 14:43:33.011966 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-accelerators-collector-config\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.011966 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.011911    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/34467f13-07d0-4d86-bdec-caa7c2098e84-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.012342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012170    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-sys\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.012342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012241    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xh5z\" (UniqueName: \"kubernetes.io/projected/35527307-227f-4f31-a7e0-fa2c7fa7e13d-kube-api-access-4xh5z\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.012342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012268    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35527307-227f-4f31-a7e0-fa2c7fa7e13d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.012342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012293    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.012342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012335    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.012575 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012351    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca37333a-5822-44cd-8482-e673c1cb7d18-metrics-client-ca\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.012575 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ngm6\" (UniqueName: \"kubernetes.io/projected/ca37333a-5822-44cd-8482-e673c1cb7d18-kube-api-access-4ngm6\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.012575 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012472    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/34467f13-07d0-4d86-bdec-caa7c2098e84-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.012851 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012826    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34467f13-07d0-4d86-bdec-caa7c2098e84-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.013022 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.012973    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-textfile\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.013407 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.013385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca37333a-5822-44cd-8482-e673c1cb7d18-sys\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.013507 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.013458    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.013590 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.013567    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-accelerators-collector-config\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.013750 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.013725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35527307-227f-4f31-a7e0-fa2c7fa7e13d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.014404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.014376    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35527307-227f-4f31-a7e0-fa2c7fa7e13d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.014589 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.014569    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.014651 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.014632    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/35527307-227f-4f31-a7e0-fa2c7fa7e13d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.015463 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.015444    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.022555 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.022524    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f6sg\" (UniqueName: \"kubernetes.io/projected/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-api-access-7f6sg\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.022854 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.022832    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ngm6\" (UniqueName: \"kubernetes.io/projected/ca37333a-5822-44cd-8482-e673c1cb7d18-kube-api-access-4ngm6\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.023439 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.023421    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xh5z\" (UniqueName: \"kubernetes.io/projected/35527307-227f-4f31-a7e0-fa2c7fa7e13d-kube-api-access-4xh5z\") pod \"openshift-state-metrics-657fd6c54-vwnfk\" (UID: \"35527307-227f-4f31-a7e0-fa2c7fa7e13d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.167685 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.167601    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"
Sep 10 14:43:33.443523 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.443493    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk"]
Sep 10 14:43:33.446068 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:33.446038    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35527307_227f_4f31_a7e0_fa2c7fa7e13d.slice/crio-8175becffa8017c6d619dc5fd86ae77558cefb2f0933c596c614633d7fbbad49 WatchSource:0}: Error finding container 8175becffa8017c6d619dc5fd86ae77558cefb2f0933c596c614633d7fbbad49: Status 404 returned error can't find the container with id 8175becffa8017c6d619dc5fd86ae77558cefb2f0933c596c614633d7fbbad49
Sep 10 14:43:33.516806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.516781    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.516918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.516881    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-tls\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.519126 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.519100    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca37333a-5822-44cd-8482-e673c1cb7d18-node-exporter-tls\") pod \"node-exporter-89hhw\" (UID: \"ca37333a-5822-44cd-8482-e673c1cb7d18\") " pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.519268 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.519250    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/34467f13-07d0-4d86-bdec-caa7c2098e84-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-k4nzs\" (UID: \"34467f13-07d0-4d86-bdec-caa7c2098e84\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.783206 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.783127    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"
Sep 10 14:43:33.804479 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.804450    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-89hhw"
Sep 10 14:43:33.815754 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:33.815723    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca37333a_5822_44cd_8482_e673c1cb7d18.slice/crio-4c0b49b4428048152f77435860042e6d07adf7c896b29c89872263688c2a3f9e WatchSource:0}: Error finding container 4c0b49b4428048152f77435860042e6d07adf7c896b29c89872263688c2a3f9e: Status 404 returned error can't find the container with id 4c0b49b4428048152f77435860042e6d07adf7c896b29c89872263688c2a3f9e
Sep 10 14:43:33.926313 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.926277    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:43:33.930967 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.930938    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:33.933802 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.933761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 14:43:33.933802 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.933781    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 14:43:33.933802 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.933761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 14:43:33.933990 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.933810    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 14:43:33.933990 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.933812    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 14:43:33.933990 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.933761    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 14:43:33.934114 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.934097    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s7mmd\""
Sep 10 14:43:33.934285 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.934267    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 14:43:33.934377 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.934322    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 14:43:33.934737 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.934723    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 14:43:33.944413 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:33.944393    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:43:34.019047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019011    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-volume\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019250 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019061    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqlfn\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-kube-api-access-tqlfn\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019250 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-web-config\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019250 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019250 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019250 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019213    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019272    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019309    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-out\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019402    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.019581 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.019464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.055827 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.055766    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs"]
Sep 10 14:43:34.058329 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:34.058296    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34467f13_07d0_4d86_bdec_caa7c2098e84.slice/crio-e23079ec76a1d7405a19d9bfc494cd952d1450e99e62395a9a2a357a1e9710ae WatchSource:0}: Error finding container e23079ec76a1d7405a19d9bfc494cd952d1450e99e62395a9a2a357a1e9710ae: Status 404 returned error can't find the container with id e23079ec76a1d7405a19d9bfc494cd952d1450e99e62395a9a2a357a1e9710ae
Sep 10 14:43:34.120656 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120620    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-web-config\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.120656 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120658    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.120865 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.120865 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.120865 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120794    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.120865 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120822    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-out\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121025 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121025 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120926    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121025 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.120951    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121025 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.121011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121203 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.121057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121203 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.121090    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-volume\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121203 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:34.121111    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle podName:b63a37bd-9303-4894-84ef-0b01e6e09ec8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:34.621086821 +0000 UTC m=+59.107774717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8") : configmap references non-existent config key: ca-bundle.crt
Sep 10 14:43:34.121203 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.121149    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqlfn\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-kube-api-access-tqlfn\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.121702 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.121647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.122050 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.122006    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.124104 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.124058    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-out\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.124723 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.124440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.124723 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.124681    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.124960 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.124940    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.125088 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.125067    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.125180 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.125149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.125262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.125244    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-volume\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.125305 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.125287    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.125546 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.125530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-web-config\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.132008 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.131986    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqlfn\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-kube-api-access-tqlfn\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.276396 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.276360    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6579f98c95-5sknz"]
Sep 10 14:43:34.280250 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.280209    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.282806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.282783    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 14:43:34.282915 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.282837    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 14:43:34.283042 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.283028    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 14:43:34.283153 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.283138    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2vndb\""
Sep 10 14:43:34.283211 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.283197    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 14:43:34.283389 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.283372    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 14:43:34.283460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.283403    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 14:43:34.283518 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.283454    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 14:43:34.289768 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.289703    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6579f98c95-5sknz"]
Sep 10 14:43:34.322998 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.322922    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lds6x\" (UniqueName: \"kubernetes.io/projected/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-kube-api-access-lds6x\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.322998 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.322963    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-oauth-config\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.323192 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.323060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-oauth-serving-cert\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.323192 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.323102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-serving-cert\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.323192 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.323154    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-config\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.323192 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.323180    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-service-ca\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.374189 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.374147    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89hhw" event={"ID":"ca37333a-5822-44cd-8482-e673c1cb7d18","Type":"ContainerStarted","Data":"4c0b49b4428048152f77435860042e6d07adf7c896b29c89872263688c2a3f9e"}
Sep 10 14:43:34.375717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.375683    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs" event={"ID":"34467f13-07d0-4d86-bdec-caa7c2098e84","Type":"ContainerStarted","Data":"e23079ec76a1d7405a19d9bfc494cd952d1450e99e62395a9a2a357a1e9710ae"}
Sep 10 14:43:34.377794 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.377765    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk" event={"ID":"35527307-227f-4f31-a7e0-fa2c7fa7e13d","Type":"ContainerStarted","Data":"3e0da0f00631c8b8a30bed9e615030b1a42e5b494f01f4e97095b7f535bdc7dc"}
Sep 10 14:43:34.377870 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.377828    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk" event={"ID":"35527307-227f-4f31-a7e0-fa2c7fa7e13d","Type":"ContainerStarted","Data":"982508af188bbadd87599f2c021d2ad9e08aa9a9298132e013e27a39f9595e73"}
Sep 10 14:43:34.377870 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.377862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk" event={"ID":"35527307-227f-4f31-a7e0-fa2c7fa7e13d","Type":"ContainerStarted","Data":"8175becffa8017c6d619dc5fd86ae77558cefb2f0933c596c614633d7fbbad49"}
Sep 10 14:43:34.423980 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.423946    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lds6x\" (UniqueName: \"kubernetes.io/projected/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-kube-api-access-lds6x\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.423980 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.423988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-oauth-config\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.424248 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.424017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-oauth-serving-cert\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.424248 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.424036    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-serving-cert\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.424248 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.424087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-config\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.424248 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.424109    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-service-ca\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.424883 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.424824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-service-ca\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.424883 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.424824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-oauth-serving-cert\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.424883 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.424865    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-config\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.427022 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.426996    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-oauth-config\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.427160 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.427135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-serving-cert\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.432428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.432384    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lds6x\" (UniqueName: \"kubernetes.io/projected/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-kube-api-access-lds6x\") pod \"console-6579f98c95-5sknz\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") " pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.591849 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.591767    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:34.626057 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.626029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.626888 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.626868    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.841457 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.841419    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:43:34.939727 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.939698    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"]
Sep 10 14:43:34.954502 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.954476    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6579f98c95-5sknz"]
Sep 10 14:43:34.954600 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.954514    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"]
Sep 10 14:43:34.954633 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.954616    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:34.957316 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.957285    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r78vf\""
Sep 10 14:43:34.957449 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.957393    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 14:43:34.957449 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.957427    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 14:43:34.957698 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.957673    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2v1c3f37luvip\""
Sep 10 14:43:34.957796 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.957695    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 14:43:34.957796 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.957698    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 14:43:34.957796 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:34.957678    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 14:43:35.030721 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ls6\" (UniqueName: \"kubernetes.io/projected/85c7d4f8-52fc-498f-91e5-bf1dab056856-kube-api-access-57ls6\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.030867 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030732    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.030867 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030776    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-grpc-tls\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.030867 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030818    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.030980 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030872    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.030980 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030920    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-tls\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.030980 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.030980 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.030972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85c7d4f8-52fc-498f-91e5-bf1dab056856-metrics-client-ca\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132114 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132114 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132076    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132119    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-tls\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132196    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85c7d4f8-52fc-498f-91e5-bf1dab056856-metrics-client-ca\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132294    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ls6\" (UniqueName: \"kubernetes.io/projected/85c7d4f8-52fc-498f-91e5-bf1dab056856-kube-api-access-57ls6\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132323    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.132373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.132361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-grpc-tls\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.133788 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.133757    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85c7d4f8-52fc-498f-91e5-bf1dab056856-metrics-client-ca\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.135781 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.135745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-tls\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.135886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.135751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-grpc-tls\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.136464 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.136437    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.136591 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.136568    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.136650 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.136628    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.138020 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.138001    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/85c7d4f8-52fc-498f-91e5-bf1dab056856-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.140918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.140878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ls6\" (UniqueName: \"kubernetes.io/projected/85c7d4f8-52fc-498f-91e5-bf1dab056856-kube-api-access-57ls6\") pod \"thanos-querier-6f98cb55f6-gkjjq\" (UID: \"85c7d4f8-52fc-498f-91e5-bf1dab056856\") " pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.265335 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.265297    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:35.289387 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:35.289351    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec7b6e2_33d3_4ddb_8f90_0f60c3397ba1.slice/crio-08d3e7d53b538ece7fe827ab3d7cab535f15ccc2296bdc8f8663fb5bb2a9bcf7 WatchSource:0}: Error finding container 08d3e7d53b538ece7fe827ab3d7cab535f15ccc2296bdc8f8663fb5bb2a9bcf7: Status 404 returned error can't find the container with id 08d3e7d53b538ece7fe827ab3d7cab535f15ccc2296bdc8f8663fb5bb2a9bcf7
Sep 10 14:43:35.385945 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.385872    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6579f98c95-5sknz" event={"ID":"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1","Type":"ContainerStarted","Data":"08d3e7d53b538ece7fe827ab3d7cab535f15ccc2296bdc8f8663fb5bb2a9bcf7"}
Sep 10 14:43:35.585170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.585140    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"]
Sep 10 14:43:35.588164 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:35.588134    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:43:36.032152 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:36.032094    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c7d4f8_52fc_498f_91e5_bf1dab056856.slice/crio-0aa346b1a8ce7154143a0137631b9767b854ef3ae531d72f3a4ccff9f215d6e6 WatchSource:0}: Error finding container 0aa346b1a8ce7154143a0137631b9767b854ef3ae531d72f3a4ccff9f215d6e6: Status 404 returned error can't find the container with id 0aa346b1a8ce7154143a0137631b9767b854ef3ae531d72f3a4ccff9f215d6e6
Sep 10 14:43:36.032854 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:36.032827    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63a37bd_9303_4894_84ef_0b01e6e09ec8.slice/crio-c869616c2a17f6c0ad9247bfafe9371fcaa6d853044eaaaf67cb4c7fdb392d42 WatchSource:0}: Error finding container c869616c2a17f6c0ad9247bfafe9371fcaa6d853044eaaaf67cb4c7fdb392d42: Status 404 returned error can't find the container with id c869616c2a17f6c0ad9247bfafe9371fcaa6d853044eaaaf67cb4c7fdb392d42
Sep 10 14:43:36.282976 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.282881    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rjdrt"
Sep 10 14:43:36.391578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.391542    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca37333a-5822-44cd-8482-e673c1cb7d18" containerID="a24897c532395f4132103c2e3ee435a9ad07228d99adc713a2c2f85eafeb0716" exitCode=0
Sep 10 14:43:36.391764 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.391627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89hhw" event={"ID":"ca37333a-5822-44cd-8482-e673c1cb7d18","Type":"ContainerDied","Data":"a24897c532395f4132103c2e3ee435a9ad07228d99adc713a2c2f85eafeb0716"}
Sep 10 14:43:36.393124 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.393098    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerStarted","Data":"c869616c2a17f6c0ad9247bfafe9371fcaa6d853044eaaaf67cb4c7fdb392d42"}
Sep 10 14:43:36.395906 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.395878    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs" event={"ID":"34467f13-07d0-4d86-bdec-caa7c2098e84","Type":"ContainerStarted","Data":"117476a98815a90448c774ddd85f37071f008c2c756b53eb1fffd186b7d9d278"}
Sep 10 14:43:36.396009 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.395937    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs" event={"ID":"34467f13-07d0-4d86-bdec-caa7c2098e84","Type":"ContainerStarted","Data":"3294e761740757490493238962b6aa61033e8b1bc690c3e495397500d0175820"}
Sep 10 14:43:36.396009 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.395959    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs" event={"ID":"34467f13-07d0-4d86-bdec-caa7c2098e84","Type":"ContainerStarted","Data":"50052b0e745a8fa5969c4e70fbfb64dc007d9435ab612b7f476040b30c91b9db"}
Sep 10 14:43:36.400469 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.400440    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk" event={"ID":"35527307-227f-4f31-a7e0-fa2c7fa7e13d","Type":"ContainerStarted","Data":"7605a0dc7133837853a68348d164d557f084cd813ff012fa5656d9df3cd1db59"}
Sep 10 14:43:36.402093 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.402068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" event={"ID":"85c7d4f8-52fc-498f-91e5-bf1dab056856","Type":"ContainerStarted","Data":"0aa346b1a8ce7154143a0137631b9767b854ef3ae531d72f3a4ccff9f215d6e6"}
Sep 10 14:43:36.428524 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.428467    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-k4nzs" podStartSLOduration=2.427773188 podStartE2EDuration="4.428446976s" totalImagesPullingTime="2.000673788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:36.42767973 +0000 UTC m=+60.914367622" watchObservedRunningTime="2026-09-10 14:43:36.428446976 +0000 UTC m=+60.915134867"
Sep 10 14:43:36.449087 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:36.449034    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-vwnfk" podStartSLOduration=2.689153065 podStartE2EDuration="4.44901286s" totalImagesPullingTime="1.759859795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:36.447893888 +0000 UTC m=+60.934581782" watchObservedRunningTime="2026-09-10 14:43:36.44901286 +0000 UTC m=+60.935700753"
Sep 10 14:43:37.268926 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.268889    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7f5544f95d-699pr"]
Sep 10 14:43:37.271572 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.271544    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.275448 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.274873    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 14:43:37.275448 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.275263    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 14:43:37.275884 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.275862    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 14:43:37.275965 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.275925    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6t6rdl4b9fe3d\""
Sep 10 14:43:37.276043 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.275871    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kpt5n\""
Sep 10 14:43:37.276749 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.276730    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 14:43:37.290971 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.285698    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f5544f95d-699pr"]
Sep 10 14:43:37.354107 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.354065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-secret-metrics-server-client-certs\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.354319 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.354155    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-client-ca-bundle\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.354319 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.354201    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-secret-metrics-server-tls\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.354319 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.354252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9e23cf93-6e32-4968-8886-3d7e58600b4a-metrics-server-audit-profiles\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.354319 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.354301    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9e23cf93-6e32-4968-8886-3d7e58600b4a-audit-log\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.354542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.354328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ln4v\" (UniqueName: \"kubernetes.io/projected/9e23cf93-6e32-4968-8886-3d7e58600b4a-kube-api-access-7ln4v\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.354542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.354360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e23cf93-6e32-4968-8886-3d7e58600b4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.411361 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.411323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89hhw" event={"ID":"ca37333a-5822-44cd-8482-e673c1cb7d18","Type":"ContainerStarted","Data":"62d44afb4ae6621aa300b295a9fdc40eabc97215d6618c87d5f95a4d2e3d7a52"}
Sep 10 14:43:37.411523 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.411381    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-89hhw" event={"ID":"ca37333a-5822-44cd-8482-e673c1cb7d18","Type":"ContainerStarted","Data":"1a40d090c0ad4d15e8b4a6e8d9fb3fc22f35af2001100ea508f7091c97ef64bf"}
Sep 10 14:43:37.430365 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.430277    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-89hhw" podStartSLOduration=3.118041354 podStartE2EDuration="5.430255503s" totalImagesPullingTime="2.220312354s" totalInitContainerRuntime="91.901795ms" isStatefulPod=true podCreationTimestamp="2026-09-10 14:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:37.429066991 +0000 UTC m=+61.915754883" watchObservedRunningTime="2026-09-10 14:43:37.430255503 +0000 UTC m=+61.916943396"
Sep 10 14:43:37.455280 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.455240    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-client-ca-bundle\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.455453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.455311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-secret-metrics-server-tls\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.455453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.455345    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9e23cf93-6e32-4968-8886-3d7e58600b4a-metrics-server-audit-profiles\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.455560 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.455515    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9e23cf93-6e32-4968-8886-3d7e58600b4a-audit-log\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.455560 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.455543    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ln4v\" (UniqueName: \"kubernetes.io/projected/9e23cf93-6e32-4968-8886-3d7e58600b4a-kube-api-access-7ln4v\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.455659 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.455591    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e23cf93-6e32-4968-8886-3d7e58600b4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.455741 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.455713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-secret-metrics-server-client-certs\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.457195 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.457160    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9e23cf93-6e32-4968-8886-3d7e58600b4a-audit-log\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.457358 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.457296    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9e23cf93-6e32-4968-8886-3d7e58600b4a-metrics-server-audit-profiles\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.457897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.457869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e23cf93-6e32-4968-8886-3d7e58600b4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.458725 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.458682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-client-ca-bundle\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.461713 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.461693    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-secret-metrics-server-client-certs\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.461825 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.461719    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9e23cf93-6e32-4968-8886-3d7e58600b4a-secret-metrics-server-tls\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.465965 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.465941    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ln4v\" (UniqueName: \"kubernetes.io/projected/9e23cf93-6e32-4968-8886-3d7e58600b4a-kube-api-access-7ln4v\") pod \"metrics-server-7f5544f95d-699pr\" (UID: \"9e23cf93-6e32-4968-8886-3d7e58600b4a\") " pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.603005 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.602911    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:37.634475 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.634445    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"]
Sep 10 14:43:37.636989 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.636965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:37.639455 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.639433    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 14:43:37.639571 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.639480    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n6jph\""
Sep 10 14:43:37.648436 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.648120    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"]
Sep 10 14:43:37.657493 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.657468    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-224x4\" (UniqueName: \"kubernetes.io/projected/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-kube-api-access-224x4\") pod \"monitoring-plugin-85bbbbbf4-h4ps8\" (UID: \"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:37.657642 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.657514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-h4ps8\" (UID: \"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:37.758549 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.758511    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-224x4\" (UniqueName: \"kubernetes.io/projected/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-kube-api-access-224x4\") pod \"monitoring-plugin-85bbbbbf4-h4ps8\" (UID: \"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:37.758549 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.758558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-h4ps8\" (UID: \"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:37.758787 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:37.758668    2558 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 14:43:37.758787 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:43:37.758730    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-monitoring-plugin-cert podName:c2dfa7b0-b5cc-4a78-a442-b157cefdb26a nodeName:}" failed. No retries permitted until 2026-09-10 14:43:38.258712144 +0000 UTC m=+62.745400014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-h4ps8" (UID: "c2dfa7b0-b5cc-4a78-a442-b157cefdb26a") : secret "monitoring-plugin-cert" not found
Sep 10 14:43:37.771249 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:37.767891    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-224x4\" (UniqueName: \"kubernetes.io/projected/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-kube-api-access-224x4\") pod \"monitoring-plugin-85bbbbbf4-h4ps8\" (UID: \"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:38.057487 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.057454    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"]
Sep 10 14:43:38.060555 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.060534    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.063243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.062975    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 14:43:38.063243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.063077    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 14:43:38.063243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.063092    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 14:43:38.063243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.063125    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pm8nz\""
Sep 10 14:43:38.063243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.063126    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 14:43:38.063243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.063079    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 14:43:38.068522 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.068500    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 14:43:38.073373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.073301    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"]
Sep 10 14:43:38.162570 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-telemeter-client-tls\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.162710 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162577    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.162710 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162645    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-secret-telemeter-client\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.162710 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4k4w\" (UniqueName: \"kubernetes.io/projected/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-kube-api-access-n4k4w\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.162820 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-federate-client-tls\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.162820 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-metrics-client-ca\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.162820 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162761    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.162911 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.162852    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-serving-certs-ca-bundle\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263518 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-secret-telemeter-client\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263527    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4k4w\" (UniqueName: \"kubernetes.io/projected/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-kube-api-access-n4k4w\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263568    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-federate-client-tls\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263592    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-metrics-client-ca\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263633    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263860 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263704    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-serving-certs-ca-bundle\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263860 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-h4ps8\" (UID: \"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:38.263860 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-telemeter-client-tls\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.263860 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.263806    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.264946 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.264871    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.265548 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.265517    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-serving-certs-ca-bundle\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.265676 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.265517    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-metrics-client-ca\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.266890 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.266860    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-secret-telemeter-client\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.266982 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.266897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.267103 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.267083    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c2dfa7b0-b5cc-4a78-a442-b157cefdb26a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-h4ps8\" (UID: \"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:38.267271 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.267253    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-federate-client-tls\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.267351 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.267303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-telemeter-client-tls\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.272019 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.271998    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4k4w\" (UniqueName: \"kubernetes.io/projected/4f9bcdb1-9d2b-41d2-8816-62117086a9c6-kube-api-access-n4k4w\") pod \"telemeter-client-5855b59cc9-g2fvt\" (UID: \"4f9bcdb1-9d2b-41d2-8816-62117086a9c6\") " pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.373516 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.373428    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"
Sep 10 14:43:38.549666 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:38.549622    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:39.610797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:39.610774    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7f5544f95d-699pr"]
Sep 10 14:43:39.615980 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:39.615950    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e23cf93_6e32_4968_8886_3d7e58600b4a.slice/crio-cb15f41756d0acafeee0b6ee68afbf4fc66054f808265bb4e5e91dd7ac37d123 WatchSource:0}: Error finding container cb15f41756d0acafeee0b6ee68afbf4fc66054f808265bb4e5e91dd7ac37d123: Status 404 returned error can't find the container with id cb15f41756d0acafeee0b6ee68afbf4fc66054f808265bb4e5e91dd7ac37d123
Sep 10 14:43:39.637710 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:39.637682    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"]
Sep 10 14:43:39.640984 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:39.640958    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2dfa7b0_b5cc_4a78_a442_b157cefdb26a.slice/crio-eca976935ea815dafbd70b3cdb3e8868e33dee4bfefa0841ce4f91390c5e21cb WatchSource:0}: Error finding container eca976935ea815dafbd70b3cdb3e8868e33dee4bfefa0841ce4f91390c5e21cb: Status 404 returned error can't find the container with id eca976935ea815dafbd70b3cdb3e8868e33dee4bfefa0841ce4f91390c5e21cb
Sep 10 14:43:39.674907 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:39.674794    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5855b59cc9-g2fvt"]
Sep 10 14:43:40.365191 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.365156    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bx5ht"
Sep 10 14:43:40.430248 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.430131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6579f98c95-5sknz" event={"ID":"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1","Type":"ContainerStarted","Data":"c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652"}
Sep 10 14:43:40.435078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.434839    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" event={"ID":"85c7d4f8-52fc-498f-91e5-bf1dab056856","Type":"ContainerStarted","Data":"86d377735c411f4674f33c0c200979fc7ed2cdb95919e6cea705d1ead4090208"}
Sep 10 14:43:40.435078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.434911    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" event={"ID":"85c7d4f8-52fc-498f-91e5-bf1dab056856","Type":"ContainerStarted","Data":"b23032a4c56fb9f027c3f590139f67a836a62a8c59a67e3b1119c02af149d2fa"}
Sep 10 14:43:40.435078 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.434997    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" event={"ID":"85c7d4f8-52fc-498f-91e5-bf1dab056856","Type":"ContainerStarted","Data":"f9d99cf6a36f72ea0a3e4b2fda3ba0ff5f39d67b38fb3159c2eccab6ccf9d4ee"}
Sep 10 14:43:40.437505 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.437418    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr" event={"ID":"9e23cf93-6e32-4968-8886-3d7e58600b4a","Type":"ContainerStarted","Data":"cb15f41756d0acafeee0b6ee68afbf4fc66054f808265bb4e5e91dd7ac37d123"}
Sep 10 14:43:40.440870 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.440703    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerID="1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93" exitCode=0
Sep 10 14:43:40.440870 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.440812    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93"}
Sep 10 14:43:40.443086 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.443006    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8" event={"ID":"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a","Type":"ContainerStarted","Data":"eca976935ea815dafbd70b3cdb3e8868e33dee4bfefa0841ce4f91390c5e21cb"}
Sep 10 14:43:40.448149 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.447304    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt" event={"ID":"4f9bcdb1-9d2b-41d2-8816-62117086a9c6","Type":"ContainerStarted","Data":"aa19a5e20e7ca7c4665fb03a9aca82cc816437a00b106b4545876d2161b25cd2"}
Sep 10 14:43:40.449623 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:40.448831    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6579f98c95-5sknz" podStartSLOduration=2.40894747 podStartE2EDuration="6.448817999s" totalImagesPullingTime="4.039870529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:40.446885721 +0000 UTC m=+64.933573625" watchObservedRunningTime="2026-09-10 14:43:40.448817999 +0000 UTC m=+64.935505890"
Sep 10 14:43:41.454411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.454372    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" event={"ID":"85c7d4f8-52fc-498f-91e5-bf1dab056856","Type":"ContainerStarted","Data":"2008ca4159b5eeb205c7ec3d946a7713486fcd647969a9e0a9751438c5e65d3d"}
Sep 10 14:43:41.454748 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.454487    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" event={"ID":"85c7d4f8-52fc-498f-91e5-bf1dab056856","Type":"ContainerStarted","Data":"ab4bd88170a382afbc84e1c392fa8d3e66add818ddb66afc34009612d2f416d7"}
Sep 10 14:43:41.702247 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.702145    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:41.705046 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.705015    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:43:41.715848 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.715820    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb75b7e-b7c0-4bfa-8c51-9b96e4449391-metrics-certs\") pod \"network-metrics-daemon-hbxnv\" (UID: \"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391\") " pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:41.803267 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.803204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:41.805883 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.805853    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:43:41.816525 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.816498    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1938126-5194-491b-b8e3-a602e268c691-original-pull-secret\") pod \"global-pull-secret-syncer-g7dg8\" (UID: \"c1938126-5194-491b-b8e3-a602e268c691\") " pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:41.825473 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.825439    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7dg8"
Sep 10 14:43:41.841822 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.841802    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnf6m\""
Sep 10 14:43:41.849997 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.849977    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbxnv"
Sep 10 14:43:41.904508 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.904477    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:41.907073 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.907052    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:43:41.917940 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.917922    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:43:41.928487 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:41.928470    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6hpv\" (UniqueName: \"kubernetes.io/projected/f0076e40-95a9-473a-bfdc-2e93c30e9f7d-kube-api-access-s6hpv\") pod \"network-check-target-jt5pz\" (UID: \"f0076e40-95a9-473a-bfdc-2e93c30e9f7d\") " pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:42.135576 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.135398    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c8srj\""
Sep 10 14:43:42.143842 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.143812    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:42.234504 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.234476    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g7dg8"]
Sep 10 14:43:42.237577 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.237551    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hbxnv"]
Sep 10 14:43:42.245743 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:42.244271    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb75b7e_b7c0_4bfa_8c51_9b96e4449391.slice/crio-952afe4765f6d719b100595fcb62f43b845d0c413743a994961cd7332e6aaa14 WatchSource:0}: Error finding container 952afe4765f6d719b100595fcb62f43b845d0c413743a994961cd7332e6aaa14: Status 404 returned error can't find the container with id 952afe4765f6d719b100595fcb62f43b845d0c413743a994961cd7332e6aaa14
Sep 10 14:43:42.435340 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.435310    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jt5pz"]
Sep 10 14:43:42.439102 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:42.439077    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0076e40_95a9_473a_bfdc_2e93c30e9f7d.slice/crio-776b83924ce7aaa549b2ee0693de8be01571525305758c691dd2123dc8fbf20f WatchSource:0}: Error finding container 776b83924ce7aaa549b2ee0693de8be01571525305758c691dd2123dc8fbf20f: Status 404 returned error can't find the container with id 776b83924ce7aaa549b2ee0693de8be01571525305758c691dd2123dc8fbf20f
Sep 10 14:43:42.459065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.458994    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g7dg8" event={"ID":"c1938126-5194-491b-b8e3-a602e268c691","Type":"ContainerStarted","Data":"498bb5441970583466485f519552aac4efa6c6f6a7f69838e39af34b171fe5ff"}
Sep 10 14:43:42.460533 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.460503    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8" event={"ID":"c2dfa7b0-b5cc-4a78-a442-b157cefdb26a","Type":"ContainerStarted","Data":"cf9e8d9336116cae4a4195bc512e08eb8e422efeeacddd2e533e2869b1d90b9b"}
Sep 10 14:43:42.460741 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.460685    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:42.465524 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.463418    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt" event={"ID":"4f9bcdb1-9d2b-41d2-8816-62117086a9c6","Type":"ContainerStarted","Data":"a0fdce26f0ef155b5891b545305428c0f0d61ee88a058f9c537758442b557da3"}
Sep 10 14:43:42.465524 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.463481    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt" event={"ID":"4f9bcdb1-9d2b-41d2-8816-62117086a9c6","Type":"ContainerStarted","Data":"05702a4b1631b89fadff62c41568168a4c8f5732edc43b7ecc1b70a84353e537"}
Sep 10 14:43:42.465524 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.463508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt" event={"ID":"4f9bcdb1-9d2b-41d2-8816-62117086a9c6","Type":"ContainerStarted","Data":"d680f65cbc448eadb7ee46f0fcb5a1e67ffb5164deb0e322d7933af2765cd5c0"}
Sep 10 14:43:42.466981 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.466953    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" event={"ID":"85c7d4f8-52fc-498f-91e5-bf1dab056856","Type":"ContainerStarted","Data":"e4098f0472c08a4981008923e26ca4c95dbd8901f85ad187f813d83a000d80cb"}
Sep 10 14:43:42.467068 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.467045    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8"
Sep 10 14:43:42.468537 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.468518    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr" event={"ID":"9e23cf93-6e32-4968-8886-3d7e58600b4a","Type":"ContainerStarted","Data":"9562a431b286bde83c68cd1d80bc303123bbe884ca4b618ca034af17a419867f"}
Sep 10 14:43:42.469764 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.469736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jt5pz" event={"ID":"f0076e40-95a9-473a-bfdc-2e93c30e9f7d","Type":"ContainerStarted","Data":"776b83924ce7aaa549b2ee0693de8be01571525305758c691dd2123dc8fbf20f"}
Sep 10 14:43:42.470841 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.470820    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hbxnv" event={"ID":"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391","Type":"ContainerStarted","Data":"952afe4765f6d719b100595fcb62f43b845d0c413743a994961cd7332e6aaa14"}
Sep 10 14:43:42.477144 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.477107    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-h4ps8" podStartSLOduration=3.171559842 podStartE2EDuration="5.477093091s" totalImagesPullingTime="2.305533249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:42.476342417 +0000 UTC m=+66.963030308" watchObservedRunningTime="2026-09-10 14:43:42.477093091 +0000 UTC m=+66.963780976"
Sep 10 14:43:42.509452 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.509397    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq" podStartSLOduration=4.272435808 podStartE2EDuration="8.509376698s" totalImagesPullingTime="4.23694089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:42.507994179 +0000 UTC m=+66.994682107" watchObservedRunningTime="2026-09-10 14:43:42.509376698 +0000 UTC m=+66.996064584"
Sep 10 14:43:42.524814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.524756    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr" podStartSLOduration=3.19484782 podStartE2EDuration="5.524734685s" totalImagesPullingTime="2.329886865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:42.524264257 +0000 UTC m=+67.010952145" watchObservedRunningTime="2026-09-10 14:43:42.524734685 +0000 UTC m=+67.011422576"
Sep 10 14:43:42.546654 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:42.546608    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5855b59cc9-g2fvt" podStartSLOduration=2.271955975 podStartE2EDuration="4.546595426s" totalImagesPullingTime="2.274639451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:42.544570415 +0000 UTC m=+67.031258310" watchObservedRunningTime="2026-09-10 14:43:42.546595426 +0000 UTC m=+67.033283317"
Sep 10 14:43:43.479548 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.479429    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:43.673797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.673764    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6579f98c95-5sknz"]
Sep 10 14:43:43.699162 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.699072    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d86459776-pspcq"]
Sep 10 14:43:43.702918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.702889    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.709757 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.709723    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d86459776-pspcq"]
Sep 10 14:43:43.712300 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.712277    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 14:43:43.819706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.819565    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-service-ca\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.819706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.819623    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-config\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.819901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.819723    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2bzg\" (UniqueName: \"kubernetes.io/projected/105a530c-0f26-4d35-b8a3-bf39dc943f52-kube-api-access-m2bzg\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.819901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.819790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-trusted-ca-bundle\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.819901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.819826    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-serving-cert\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.819901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.819862    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-oauth-serving-cert\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.820048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.819908    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-oauth-config\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.920866 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.920834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2bzg\" (UniqueName: \"kubernetes.io/projected/105a530c-0f26-4d35-b8a3-bf39dc943f52-kube-api-access-m2bzg\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.921004 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.920912    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-trusted-ca-bundle\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.921004 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.920947    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-serving-cert\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.921004 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.920984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-oauth-serving-cert\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.921170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.921029    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-oauth-config\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.921170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.921067    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-service-ca\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.921170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.921104    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-config\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.922383 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.921869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-config\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.922383 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.921874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-trusted-ca-bundle\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.922383 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.922071    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-oauth-serving-cert\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.922655 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.922636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-service-ca\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.932499 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.929149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-oauth-config\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.932499 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.929494    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-serving-cert\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:43.936144 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:43.936059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2bzg\" (UniqueName: \"kubernetes.io/projected/105a530c-0f26-4d35-b8a3-bf39dc943f52-kube-api-access-m2bzg\") pod \"console-d86459776-pspcq\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") " pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:44.023831 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.023581    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:44.323648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.323617    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d86459776-pspcq"]
Sep 10 14:43:44.330826 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:43:44.330387    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105a530c_0f26_4d35_b8a3_bf39dc943f52.slice/crio-e5ee4e3894831e3e743e9aeb3208a43f3863f4fb5929c96864a1877ac7a8f945 WatchSource:0}: Error finding container e5ee4e3894831e3e743e9aeb3208a43f3863f4fb5929c96864a1877ac7a8f945: Status 404 returned error can't find the container with id e5ee4e3894831e3e743e9aeb3208a43f3863f4fb5929c96864a1877ac7a8f945
Sep 10 14:43:44.491950 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.491906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hbxnv" event={"ID":"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391","Type":"ContainerStarted","Data":"87b8edec227a9b699ef9159c4f0328dcf3298ed62814ab5a793ec77039f8c442"}
Sep 10 14:43:44.492415 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.491984    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hbxnv" event={"ID":"0fb75b7e-b7c0-4bfa-8c51-9b96e4449391","Type":"ContainerStarted","Data":"8b0b032359e98578449533cb166c234b1f7dbf2a8577578402299b5b28fb035d"}
Sep 10 14:43:44.499183 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.498410    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerStarted","Data":"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"}
Sep 10 14:43:44.499183 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.498468    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerStarted","Data":"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"}
Sep 10 14:43:44.499183 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.498496    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerStarted","Data":"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"}
Sep 10 14:43:44.499183 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.498532    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerStarted","Data":"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"}
Sep 10 14:43:44.499183 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.498554    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerStarted","Data":"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"}
Sep 10 14:43:44.499183 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.498576    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerStarted","Data":"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"}
Sep 10 14:43:44.501953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.501548    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d86459776-pspcq" event={"ID":"105a530c-0f26-4d35-b8a3-bf39dc943f52","Type":"ContainerStarted","Data":"d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4"}
Sep 10 14:43:44.501953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.501580    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d86459776-pspcq" event={"ID":"105a530c-0f26-4d35-b8a3-bf39dc943f52","Type":"ContainerStarted","Data":"e5ee4e3894831e3e743e9aeb3208a43f3863f4fb5929c96864a1877ac7a8f945"}
Sep 10 14:43:44.508420 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.508058    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6f98cb55f6-gkjjq"
Sep 10 14:43:44.509645 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.509594    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hbxnv" podStartSLOduration=66.92769058 podStartE2EDuration="1m8.509576685s" totalImagesPullingTime="1.581886105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:44.50863338 +0000 UTC m=+68.995321270" watchObservedRunningTime="2026-09-10 14:43:44.509576685 +0000 UTC m=+68.996264575"
Sep 10 14:43:44.529481 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.529393    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d86459776-pspcq" podStartSLOduration=1.529373944 podStartE2EDuration="1.529373944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:44.528801573 +0000 UTC m=+69.015489463" watchObservedRunningTime="2026-09-10 14:43:44.529373944 +0000 UTC m=+69.016061835"
Sep 10 14:43:44.593168 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.592940    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:43:44.607182 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:44.607132    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.915725351 podStartE2EDuration="11.607115764s" totalImagesPullingTime="6.679541141s" totalInitContainerRuntime="11.849272ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:44.605454665 +0000 UTC m=+69.092142557" watchObservedRunningTime="2026-09-10 14:43:44.607115764 +0000 UTC m=+69.093803654"
Sep 10 14:43:48.521846 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:48.521808    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jt5pz" event={"ID":"f0076e40-95a9-473a-bfdc-2e93c30e9f7d","Type":"ContainerStarted","Data":"cd7f8f01f9b69b2394698a978fcf11f58897cb88570ab74f42bfeceb3b8ac53f"}
Sep 10 14:43:48.522330 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:48.521876    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:43:48.523406 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:48.523382    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g7dg8" event={"ID":"c1938126-5194-491b-b8e3-a602e268c691","Type":"ContainerStarted","Data":"5228e94f32c715b58cce71f95f99ed0d5f58e674d2a52d194c7f94b96b273674"}
Sep 10 14:43:48.536705 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:48.536655    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jt5pz" podStartSLOduration=67.003088909 podStartE2EDuration="1m12.536641238s" totalImagesPullingTime="5.533552329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:48.536270645 +0000 UTC m=+73.022958535" watchObservedRunningTime="2026-09-10 14:43:48.536641238 +0000 UTC m=+73.023329126"
Sep 10 14:43:48.549803 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:48.549729    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g7dg8" podStartSLOduration=66.80865524 podStartE2EDuration="1m12.549712935s" totalImagesPullingTime="5.741057695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:48.549383054 +0000 UTC m=+73.036070944" watchObservedRunningTime="2026-09-10 14:43:48.549712935 +0000 UTC m=+73.036400808"
Sep 10 14:43:54.023815 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:54.023783    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:54.024321 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:54.023849    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:54.029095 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:54.029070    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:54.550658 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:54.550630    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:43:57.603287 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:57.603139    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:43:57.603287 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:43:57.603250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:44:08.701578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:08.701528    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6579f98c95-5sknz" podUID="bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" containerName="console" containerID="cri-o://c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652" gracePeriod=15
Sep 10 14:44:08.938687 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:08.938661    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:44:09.050466 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050392    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-oauth-serving-cert\") pod \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") "
Sep 10 14:44:09.050466 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050458    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-oauth-config\") pod \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") "
Sep 10 14:44:09.050632 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050487    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-serving-cert\") pod \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") "
Sep 10 14:44:09.050632 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050514    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-service-ca\") pod \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") "
Sep 10 14:44:09.050632 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050563    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-config\") pod \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") "
Sep 10 14:44:09.050632 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050592    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lds6x\" (UniqueName: \"kubernetes.io/projected/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-kube-api-access-lds6x\") pod \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\" (UID: \"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1\") "
Sep 10 14:44:09.050913 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050874    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" (UID: "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:44:09.051036 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050905    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-config" (OuterVolumeSpecName: "console-config") pod "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" (UID: "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:44:09.051036 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.050939    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-service-ca" (OuterVolumeSpecName: "service-ca") pod "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" (UID: "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:44:09.053011 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.052975    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" (UID: "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:09.053110 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.053007    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-kube-api-access-lds6x" (OuterVolumeSpecName: "kube-api-access-lds6x") pod "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" (UID: "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1"). InnerVolumeSpecName "kube-api-access-lds6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:44:09.053110 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.053025    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" (UID: "bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:09.152058 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.152008    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:09.152058 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.152052    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:09.152058 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.152062    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lds6x\" (UniqueName: \"kubernetes.io/projected/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-kube-api-access-lds6x\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:09.152321 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.152142    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-oauth-serving-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:09.152321 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.152152    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-oauth-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:09.152321 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.152160    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1-console-serving-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:09.608791 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.608756    2558 generic.go:356] "Generic (PLEG): container finished" podID="bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" containerID="c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652" exitCode=0
Sep 10 14:44:09.608987 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.608803    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6579f98c95-5sknz"
Sep 10 14:44:09.608987 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.608818    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6579f98c95-5sknz" event={"ID":"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1","Type":"ContainerDied","Data":"c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652"}
Sep 10 14:44:09.608987 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.608868    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6579f98c95-5sknz" event={"ID":"bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1","Type":"ContainerDied","Data":"08d3e7d53b538ece7fe827ab3d7cab535f15ccc2296bdc8f8663fb5bb2a9bcf7"}
Sep 10 14:44:09.608987 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.608894    2558 scope.go:117] "RemoveContainer" containerID="c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652"
Sep 10 14:44:09.617514 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.617493    2558 scope.go:117] "RemoveContainer" containerID="c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652"
Sep 10 14:44:09.617795 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:09.617776    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652\": container with ID starting with c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652 not found: ID does not exist" containerID="c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652"
Sep 10 14:44:09.617864 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.617805    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652"} err="failed to get container status \"c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652\": rpc error: code = NotFound desc = could not find container \"c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652\": container with ID starting with c93a5a9546efe996ed94fcf1ae0c264ca7fe2183a06c9315561341111a0db652 not found: ID does not exist"
Sep 10 14:44:09.628288 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.628235    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6579f98c95-5sknz"]
Sep 10 14:44:09.630655 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:09.630634    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6579f98c95-5sknz"]
Sep 10 14:44:10.017659 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:10.017629    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" path="/var/lib/kubelet/pods/bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1/volumes"
Sep 10 14:44:17.608120 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:17.608086    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:44:17.613106 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:17.613081    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7f5544f95d-699pr"
Sep 10 14:44:19.530634 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:19.530592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jt5pz"
Sep 10 14:44:53.208832 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.208803    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:44:53.209353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.209056    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="alertmanager" containerID="cri-o://cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5" gracePeriod=120
Sep 10 14:44:53.209353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.209124    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-metric" containerID="cri-o://0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf" gracePeriod=120
Sep 10 14:44:53.209353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.209148    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy" containerID="cri-o://69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4" gracePeriod=120
Sep 10 14:44:53.209353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.209166    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="config-reloader" containerID="cri-o://107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3" gracePeriod=120
Sep 10 14:44:53.209353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.209248    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-web" containerID="cri-o://5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52" gracePeriod=120
Sep 10 14:44:53.209353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.209182    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="prom-label-proxy" containerID="cri-o://685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823" gracePeriod=120
Sep 10 14:44:53.786798 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786766    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerID="685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823" exitCode=0
Sep 10 14:44:53.786798 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786790    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerID="69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4" exitCode=0
Sep 10 14:44:53.786798 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786796    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerID="107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3" exitCode=0
Sep 10 14:44:53.786798 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786803    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerID="cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5" exitCode=0
Sep 10 14:44:53.787048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786837    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"}
Sep 10 14:44:53.787048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"}
Sep 10 14:44:53.787048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786935    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"}
Sep 10 14:44:53.787048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:53.786960    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"}
Sep 10 14:44:54.110158 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.110074    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-664f46c5dd-wxbmj"]
Sep 10 14:44:54.110464 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.110451    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" containerName="console"
Sep 10 14:44:54.110509 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.110466    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" containerName="console"
Sep 10 14:44:54.110542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.110515    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bec7b6e2-33d3-4ddb-8f90-0f60c3397ba1" containerName="console"
Sep 10 14:44:54.112061 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.112047    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.122898 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.122874    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664f46c5dd-wxbmj"]
Sep 10 14:44:54.213631 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.213596    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-serving-cert\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.213631 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.213628    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwx6l\" (UniqueName: \"kubernetes.io/projected/296457b2-459e-429c-9011-2a6a61cd05ea-kube-api-access-lwx6l\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.214065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.213648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-service-ca\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.214065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.213672    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-oauth-serving-cert\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.214065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.213758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-trusted-ca-bundle\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.214065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.213804    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-console-config\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.214065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.213870    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-oauth-config\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.314616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.314576    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-serving-cert\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.314616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.314611    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwx6l\" (UniqueName: \"kubernetes.io/projected/296457b2-459e-429c-9011-2a6a61cd05ea-kube-api-access-lwx6l\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.314859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.314631    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-service-ca\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.314859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.314667    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-oauth-serving-cert\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.314859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.314708    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-trusted-ca-bundle\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.314859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.314753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-console-config\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.314859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.314800    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-oauth-config\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.315521 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.315490    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-service-ca\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.315666 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.315549    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-console-config\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.315732 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.315719    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-oauth-serving-cert\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.316159 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.316124    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-trusted-ca-bundle\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.317385 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.317358    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-oauth-config\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.317786 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.317767    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-serving-cert\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.322866 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.322839    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwx6l\" (UniqueName: \"kubernetes.io/projected/296457b2-459e-429c-9011-2a6a61cd05ea-kube-api-access-lwx6l\") pod \"console-664f46c5dd-wxbmj\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") " pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.421128 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.421036    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:44:54.571783 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.571753    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:54.689913 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.689886    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664f46c5dd-wxbmj"]
Sep 10 14:44:54.692350 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:44:54.692324    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296457b2_459e_429c_9011_2a6a61cd05ea.slice/crio-8df45e646acac63998744665bcccce1bea226693b3251199d3fde048d5a9559e WatchSource:0}: Error finding container 8df45e646acac63998744665bcccce1bea226693b3251199d3fde048d5a9559e: Status 404 returned error can't find the container with id 8df45e646acac63998744665bcccce1bea226693b3251199d3fde048d5a9559e
Sep 10 14:44:54.717603 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717580    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-web-config\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717711 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717617    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-web\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717711 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717639    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-cluster-tls-config\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717711 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717672    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-out\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717711 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717699    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-tls-assets\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717730    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717756    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqlfn\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-kube-api-access-tqlfn\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717783    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-main-db\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717813    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717857    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-main-tls\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.717892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.717892    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-volume\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.718880 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.718800    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:44:54.721065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.721035    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-volume" (OuterVolumeSpecName: "config-volume") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:54.721170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.721133    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:54.721638 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.721344    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:54.721638 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.721598    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:44:54.722205 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.721926    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-metrics-client-ca\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.722205 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.721989    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\" (UID: \"b63a37bd-9303-4894-84ef-0b01e6e09ec8\") "
Sep 10 14:44:54.722205 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.722069    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:44:54.722477 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.722457    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.722529 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.722485    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-tls-assets\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.722529 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.722502    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.722529 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.722516    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-alertmanager-main-db\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.722654 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.722537    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.722654 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.722552    2558 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-volume\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.723429 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.723358    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:54.723429 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.723391    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:44:54.726322 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.725331    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:54.726322 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.725624    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-out" (OuterVolumeSpecName: "config-out") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:44:54.727271 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.727102    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:54.727271 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.727234    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-kube-api-access-tqlfn" (OuterVolumeSpecName: "kube-api-access-tqlfn") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "kube-api-access-tqlfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:44:54.735875 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.735853    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-web-config" (OuterVolumeSpecName: "web-config") pod "b63a37bd-9303-4894-84ef-0b01e6e09ec8" (UID: "b63a37bd-9303-4894-84ef-0b01e6e09ec8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:44:54.793320 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.793272    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f46c5dd-wxbmj" event={"ID":"296457b2-459e-429c-9011-2a6a61cd05ea","Type":"ContainerStarted","Data":"777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221"}
Sep 10 14:44:54.793499 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.793366    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f46c5dd-wxbmj" event={"ID":"296457b2-459e-429c-9011-2a6a61cd05ea","Type":"ContainerStarted","Data":"8df45e646acac63998744665bcccce1bea226693b3251199d3fde048d5a9559e"}
Sep 10 14:44:54.796944 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.796919    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerID="0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf" exitCode=0
Sep 10 14:44:54.796944 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.796943    2558 generic.go:356] "Generic (PLEG): container finished" podID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerID="5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52" exitCode=0
Sep 10 14:44:54.797131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.796990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"}
Sep 10 14:44:54.797131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.797025    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:54.797131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.797061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"}
Sep 10 14:44:54.797131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.797104    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b63a37bd-9303-4894-84ef-0b01e6e09ec8","Type":"ContainerDied","Data":"c869616c2a17f6c0ad9247bfafe9371fcaa6d853044eaaaf67cb4c7fdb392d42"}
Sep 10 14:44:54.797131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.797125    2558 scope.go:117] "RemoveContainer" containerID="685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"
Sep 10 14:44:54.805977 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.805958    2558 scope.go:117] "RemoveContainer" containerID="0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"
Sep 10 14:44:54.811040 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.810985    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-664f46c5dd-wxbmj" podStartSLOduration=0.810964207 podStartE2EDuration="810.964207ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:44:54.810636387 +0000 UTC m=+139.297324279" watchObservedRunningTime="2026-09-10 14:44:54.810964207 +0000 UTC m=+139.297652099"
Sep 10 14:44:54.817882 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.817853    2558 scope.go:117] "RemoveContainer" containerID="69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"
Sep 10 14:44:54.823810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.823691    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqlfn\" (UniqueName: \"kubernetes.io/projected/b63a37bd-9303-4894-84ef-0b01e6e09ec8-kube-api-access-tqlfn\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.823810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.823719    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-main-tls\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.823810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.823735    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b63a37bd-9303-4894-84ef-0b01e6e09ec8-metrics-client-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.823810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.823747    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.823810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.823760    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-web-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.823810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.823772    2558 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b63a37bd-9303-4894-84ef-0b01e6e09ec8-cluster-tls-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.823810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.823782    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b63a37bd-9303-4894-84ef-0b01e6e09ec8-config-out\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:44:54.836534 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.826145    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:44:54.836534 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.827630    2558 scope.go:117] "RemoveContainer" containerID="5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"
Sep 10 14:44:54.838405 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.838042    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:44:54.850650 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.850624    2558 scope.go:117] "RemoveContainer" containerID="107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"
Sep 10 14:44:54.859433 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.859412    2558 scope.go:117] "RemoveContainer" containerID="cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"
Sep 10 14:44:54.860565 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.860545    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:44:54.860970 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.860952    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="prom-label-proxy"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.860972    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="prom-label-proxy"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.860983    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="init-config-reloader"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.860989    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="init-config-reloader"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.860997    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-metric"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861004    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-metric"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861013    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="config-reloader"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861019    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="config-reloader"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861028    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861033    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861043    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="alertmanager"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861048    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="alertmanager"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861056    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-web"
Sep 10 14:44:54.861089 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861061    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-web"
Sep 10 14:44:54.861616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861128    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="alertmanager"
Sep 10 14:44:54.861616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861143    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-web"
Sep 10 14:44:54.861616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861152    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy"
Sep 10 14:44:54.861616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861161    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="prom-label-proxy"
Sep 10 14:44:54.861616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861169    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="config-reloader"
Sep 10 14:44:54.861616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.861176    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" containerName="kube-rbac-proxy-metric"
Sep 10 14:44:54.864779 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.864760    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:54.867257 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867237    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 14:44:54.867361 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867282    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 14:44:54.867417 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867375    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 14:44:54.867527 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867496    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 14:44:54.867627 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867568    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 14:44:54.867627 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867587    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 14:44:54.867627 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867616    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s7mmd\""
Sep 10 14:44:54.867627 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867620    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 14:44:54.867833 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.867677    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 14:44:54.869058 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.869038    2558 scope.go:117] "RemoveContainer" containerID="1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93"
Sep 10 14:44:54.874704 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.874677    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 14:44:54.878433 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.878417    2558 scope.go:117] "RemoveContainer" containerID="685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"
Sep 10 14:44:54.878742 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:54.878722    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823\": container with ID starting with 685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823 not found: ID does not exist" containerID="685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"
Sep 10 14:44:54.878791 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.878752    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"} err="failed to get container status \"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823\": rpc error: code = NotFound desc = could not find container \"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823\": container with ID starting with 685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823 not found: ID does not exist"
Sep 10 14:44:54.878791 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.878770    2558 scope.go:117] "RemoveContainer" containerID="0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"
Sep 10 14:44:54.879013 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:54.878996    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf\": container with ID starting with 0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf not found: ID does not exist" containerID="0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"
Sep 10 14:44:54.879072 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879022    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"} err="failed to get container status \"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf\": rpc error: code = NotFound desc = could not find container \"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf\": container with ID starting with 0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf not found: ID does not exist"
Sep 10 14:44:54.879072 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879044    2558 scope.go:117] "RemoveContainer" containerID="69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"
Sep 10 14:44:54.879312 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:54.879295    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4\": container with ID starting with 69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4 not found: ID does not exist" containerID="69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"
Sep 10 14:44:54.879379 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879316    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"} err="failed to get container status \"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4\": rpc error: code = NotFound desc = could not find container \"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4\": container with ID starting with 69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4 not found: ID does not exist"
Sep 10 14:44:54.879434 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879380    2558 scope.go:117] "RemoveContainer" containerID="5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"
Sep 10 14:44:54.879625 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:54.879612    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52\": container with ID starting with 5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52 not found: ID does not exist" containerID="5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"
Sep 10 14:44:54.879691 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879629    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"} err="failed to get container status \"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52\": rpc error: code = NotFound desc = could not find container \"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52\": container with ID starting with 5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52 not found: ID does not exist"
Sep 10 14:44:54.879691 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879641    2558 scope.go:117] "RemoveContainer" containerID="107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"
Sep 10 14:44:54.879872 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:54.879855    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3\": container with ID starting with 107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3 not found: ID does not exist" containerID="107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"
Sep 10 14:44:54.879916 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879876    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"} err="failed to get container status \"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3\": rpc error: code = NotFound desc = could not find container \"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3\": container with ID starting with 107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3 not found: ID does not exist"
Sep 10 14:44:54.879916 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.879889    2558 scope.go:117] "RemoveContainer" containerID="cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"
Sep 10 14:44:54.880138 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:54.880121    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5\": container with ID starting with cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5 not found: ID does not exist" containerID="cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"
Sep 10 14:44:54.880179 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.880143    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"} err="failed to get container status \"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5\": rpc error: code = NotFound desc = could not find container \"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5\": container with ID starting with cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5 not found: ID does not exist"
Sep 10 14:44:54.880179 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.880160    2558 scope.go:117] "RemoveContainer" containerID="1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93"
Sep 10 14:44:54.880458 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:44:54.880441    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93\": container with ID starting with 1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93 not found: ID does not exist" containerID="1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93"
Sep 10 14:44:54.880501 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.880462    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93"} err="failed to get container status \"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93\": rpc error: code = NotFound desc = could not find container \"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93\": container with ID starting with 1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93 not found: ID does not exist"
Sep 10 14:44:54.880501 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.880475    2558 scope.go:117] "RemoveContainer" containerID="685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"
Sep 10 14:44:54.880728 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.880712    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823"} err="failed to get container status \"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823\": rpc error: code = NotFound desc = could not find container \"685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823\": container with ID starting with 685e8cbb5893314e09dde15c09be932452f09847014227cffc9c34568d19f823 not found: ID does not exist"
Sep 10 14:44:54.880777 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.880764    2558 scope.go:117] "RemoveContainer" containerID="0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"
Sep 10 14:44:54.881001 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.880981    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf"} err="failed to get container status \"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf\": rpc error: code = NotFound desc = could not find container \"0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf\": container with ID starting with 0e49971f8bd30a5d5a8bacfec047e018e3756ff9f89ae815825328a3d045d2cf not found: ID does not exist"
Sep 10 14:44:54.881041 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.881004    2558 scope.go:117] "RemoveContainer" containerID="69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"
Sep 10 14:44:54.881497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.881425    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4"} err="failed to get container status \"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4\": rpc error: code = NotFound desc = could not find container \"69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4\": container with ID starting with 69ccb1e69c5cd3f23f65a5f3689116b92d97552547e231e7076e15b39e28faf4 not found: ID does not exist"
Sep 10 14:44:54.881497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.881452    2558 scope.go:117] "RemoveContainer" containerID="5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"
Sep 10 14:44:54.882039 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.882010    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52"} err="failed to get container status \"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52\": rpc error: code = NotFound desc = could not find container \"5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52\": container with ID starting with 5c1ed642ae3e3d7360e237dc3b006ae448f6b365d9a70e3f94b2b64b7ef32c52 not found: ID does not exist"
Sep 10 14:44:54.882039 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.882039    2558 scope.go:117] "RemoveContainer" containerID="107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"
Sep 10 14:44:54.882450 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.882431    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3"} err="failed to get container status \"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3\": rpc error: code = NotFound desc = could not find container \"107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3\": container with ID starting with 107afec4a38d57a70b0444df6535216bbe7dc33af3af5a9de806eaa931d8a5f3 not found: ID does not exist"
Sep 10 14:44:54.882450 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.882450    2558 scope.go:117] "RemoveContainer" containerID="cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"
Sep 10 14:44:54.882856 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.882779    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5"} err="failed to get container status \"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5\": rpc error: code = NotFound desc = could not find container \"cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5\": container with ID starting with cc252d54bff68fa4a6051028a687acc4d0972f4afa42348f9a3b788b6c8c5ac5 not found: ID does not exist"
Sep 10 14:44:54.882856 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.882805    2558 scope.go:117] "RemoveContainer" containerID="1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93"
Sep 10 14:44:54.883177 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.883144    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93"} err="failed to get container status \"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93\": rpc error: code = NotFound desc = could not find container \"1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93\": container with ID starting with 1bcb3700db9848388193816a7afde5d1c9c76edefd6014c275b9ddc5228d5c93 not found: ID does not exist"
Sep 10 14:44:54.883634 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:54.883612    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:44:55.025999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.025916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29p62\" (UniqueName: \"kubernetes.io/projected/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-kube-api-access-29p62\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.025999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.025950    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.025999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.025969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.025999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.025987    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-web-config\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026058    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026097    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026142    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026182    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026212    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026263    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026279    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-config-out\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.026462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.026296    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.127723 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127683    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.127897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127743    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29p62\" (UniqueName: \"kubernetes.io/projected/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-kube-api-access-29p62\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.127897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127776    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.127897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127799    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.127897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127823    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-web-config\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.127897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127859    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127903    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127924    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.127962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.128020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.128064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.128086    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.128115    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-config-out\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128620 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.128588    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.128745 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.128652    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.129101 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.128953    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131698 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-config-out\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131698 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131657    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131698 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131665    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131698 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131661    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-config-volume\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131866    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-web-config\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131916    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.131956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.131919    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.133595 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.133575    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.135663 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.135641    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29p62\" (UniqueName: \"kubernetes.io/projected/fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363-kube-api-access-29p62\") pod \"alertmanager-main-0\" (UID: \"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.177680 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.177648    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:44:55.460276 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.460251    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:44:55.462439 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:44:55.462409    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4a63ef_f88e_4d50_b8e3_5d3c4c85f363.slice/crio-d92bc82262bb30b33760a4784c8de07675431d7baa28a5d91504a029714251b4 WatchSource:0}: Error finding container d92bc82262bb30b33760a4784c8de07675431d7baa28a5d91504a029714251b4: Status 404 returned error can't find the container with id d92bc82262bb30b33760a4784c8de07675431d7baa28a5d91504a029714251b4
Sep 10 14:44:55.803849 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.803770    2558 generic.go:356] "Generic (PLEG): container finished" podID="fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363" containerID="6b3277a1aa6bd75e66a14a107df7520fcf32208a7c4fe4638203b371a85e626c" exitCode=0
Sep 10 14:44:55.803985 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.803860    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerDied","Data":"6b3277a1aa6bd75e66a14a107df7520fcf32208a7c4fe4638203b371a85e626c"}
Sep 10 14:44:55.803985 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:55.803944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerStarted","Data":"d92bc82262bb30b33760a4784c8de07675431d7baa28a5d91504a029714251b4"}
Sep 10 14:44:56.022363 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.022332    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b63a37bd-9303-4894-84ef-0b01e6e09ec8" path="/var/lib/kubelet/pods/b63a37bd-9303-4894-84ef-0b01e6e09ec8/volumes"
Sep 10 14:44:56.811193 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.811159    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerStarted","Data":"501c78eb2822aa9b80cdfbdb3c373259275a0641b20c9b6b5730fc876a68b9e3"}
Sep 10 14:44:56.811699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.811240    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerStarted","Data":"7361114ccf52e03f8c18cf77c8ffb9f6e0c146fe292e0870934b3597f1444610"}
Sep 10 14:44:56.811699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.811271    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerStarted","Data":"ca63aa8927864cdcdf4b59d23320b355c14cf498623d44d8fb0e29d94bb68b84"}
Sep 10 14:44:56.811699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.811298    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerStarted","Data":"6a3fcd387739b37d8087f8ce6a60beeac5e770a90e4d2d35f7a884bfda076d6a"}
Sep 10 14:44:56.811699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.811322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerStarted","Data":"cd0fd1e2a628a15d32ac1cb5ee0bd9b824aea8f60c151d437e93642531a153b8"}
Sep 10 14:44:56.811699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.811346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363","Type":"ContainerStarted","Data":"63c812c25aad45df4fddfcaf524dd1d1de353e12c2a5bf0198a83a70dbd74aca"}
Sep 10 14:44:56.838660 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:44:56.838599    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.822894566 podStartE2EDuration="2.838581396s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.68683ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:44:56.836100645 +0000 UTC m=+141.322788536" watchObservedRunningTime="2026-09-10 14:44:56.838581396 +0000 UTC m=+141.325269286"
Sep 10 14:45:04.422177 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:04.422135    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:45:04.422591 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:04.422240    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:45:04.426934 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:04.426912    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:45:04.850556 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:04.850523    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:45:04.896765 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:04.896731    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d86459776-pspcq"]
Sep 10 14:45:29.917830 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:29.917714    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d86459776-pspcq" podUID="105a530c-0f26-4d35-b8a3-bf39dc943f52" containerName="console" containerID="cri-o://d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4" gracePeriod=15
Sep 10 14:45:30.158825 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.158798    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:45:30.227541 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.227504    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-trusted-ca-bundle\") pod \"105a530c-0f26-4d35-b8a3-bf39dc943f52\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") "
Sep 10 14:45:30.227727 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.227551    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-config\") pod \"105a530c-0f26-4d35-b8a3-bf39dc943f52\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") "
Sep 10 14:45:30.227727 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.227578    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-oauth-serving-cert\") pod \"105a530c-0f26-4d35-b8a3-bf39dc943f52\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") "
Sep 10 14:45:30.227727 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.227614    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-serving-cert\") pod \"105a530c-0f26-4d35-b8a3-bf39dc943f52\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") "
Sep 10 14:45:30.227727 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.227668    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-oauth-config\") pod \"105a530c-0f26-4d35-b8a3-bf39dc943f52\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") "
Sep 10 14:45:30.227727 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.227694    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2bzg\" (UniqueName: \"kubernetes.io/projected/105a530c-0f26-4d35-b8a3-bf39dc943f52-kube-api-access-m2bzg\") pod \"105a530c-0f26-4d35-b8a3-bf39dc943f52\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") "
Sep 10 14:45:30.227727 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.227718    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-service-ca\") pod \"105a530c-0f26-4d35-b8a3-bf39dc943f52\" (UID: \"105a530c-0f26-4d35-b8a3-bf39dc943f52\") "
Sep 10 14:45:30.228125 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.228020    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-config" (OuterVolumeSpecName: "console-config") pod "105a530c-0f26-4d35-b8a3-bf39dc943f52" (UID: "105a530c-0f26-4d35-b8a3-bf39dc943f52"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:45:30.228125 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.228026    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "105a530c-0f26-4d35-b8a3-bf39dc943f52" (UID: "105a530c-0f26-4d35-b8a3-bf39dc943f52"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:45:30.228353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.228130    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "105a530c-0f26-4d35-b8a3-bf39dc943f52" (UID: "105a530c-0f26-4d35-b8a3-bf39dc943f52"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:45:30.228353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.228204    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-service-ca" (OuterVolumeSpecName: "service-ca") pod "105a530c-0f26-4d35-b8a3-bf39dc943f52" (UID: "105a530c-0f26-4d35-b8a3-bf39dc943f52"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:45:30.230064 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.230037    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "105a530c-0f26-4d35-b8a3-bf39dc943f52" (UID: "105a530c-0f26-4d35-b8a3-bf39dc943f52"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:45:30.230175 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.230100    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/105a530c-0f26-4d35-b8a3-bf39dc943f52-kube-api-access-m2bzg" (OuterVolumeSpecName: "kube-api-access-m2bzg") pod "105a530c-0f26-4d35-b8a3-bf39dc943f52" (UID: "105a530c-0f26-4d35-b8a3-bf39dc943f52"). InnerVolumeSpecName "kube-api-access-m2bzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:45:30.230175 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.230112    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "105a530c-0f26-4d35-b8a3-bf39dc943f52" (UID: "105a530c-0f26-4d35-b8a3-bf39dc943f52"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:45:30.329231 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.329192    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-oauth-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:30.329394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.329242    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2bzg\" (UniqueName: \"kubernetes.io/projected/105a530c-0f26-4d35-b8a3-bf39dc943f52-kube-api-access-m2bzg\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:30.329394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.329252    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:30.329394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.329261    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-trusted-ca-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:30.329394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.329270    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:30.329394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.329278    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/105a530c-0f26-4d35-b8a3-bf39dc943f52-oauth-serving-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:30.329394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.329286    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/105a530c-0f26-4d35-b8a3-bf39dc943f52-console-serving-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:45:30.951572 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.951537    2558 generic.go:356] "Generic (PLEG): container finished" podID="105a530c-0f26-4d35-b8a3-bf39dc943f52" containerID="d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4" exitCode=0
Sep 10 14:45:30.952013 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.951580    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d86459776-pspcq"
Sep 10 14:45:30.952013 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.951626    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d86459776-pspcq" event={"ID":"105a530c-0f26-4d35-b8a3-bf39dc943f52","Type":"ContainerDied","Data":"d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4"}
Sep 10 14:45:30.952013 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.951672    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d86459776-pspcq" event={"ID":"105a530c-0f26-4d35-b8a3-bf39dc943f52","Type":"ContainerDied","Data":"e5ee4e3894831e3e743e9aeb3208a43f3863f4fb5929c96864a1877ac7a8f945"}
Sep 10 14:45:30.952013 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.951692    2558 scope.go:117] "RemoveContainer" containerID="d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4"
Sep 10 14:45:30.960477 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.960461    2558 scope.go:117] "RemoveContainer" containerID="d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4"
Sep 10 14:45:30.960734 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:45:30.960719    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4\": container with ID starting with d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4 not found: ID does not exist" containerID="d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4"
Sep 10 14:45:30.960806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.960740    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4"} err="failed to get container status \"d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4\": rpc error: code = NotFound desc = could not find container \"d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4\": container with ID starting with d63d8bdd4e1fa100badfb3c4c2ce47f32566115f43db6c475d2de86df8495ea4 not found: ID does not exist"
Sep 10 14:45:30.972381 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.972341    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d86459776-pspcq"]
Sep 10 14:45:30.975715 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:30.975692    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d86459776-pspcq"]
Sep 10 14:45:32.017563 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:45:32.017523    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="105a530c-0f26-4d35-b8a3-bf39dc943f52" path="/var/lib/kubelet/pods/105a530c-0f26-4d35-b8a3-bf39dc943f52/volumes"
Sep 10 14:46:03.666174 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.666118    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"]
Sep 10 14:46:03.666697 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.666440    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="105a530c-0f26-4d35-b8a3-bf39dc943f52" containerName="console"
Sep 10 14:46:03.666697 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.666453    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="105a530c-0f26-4d35-b8a3-bf39dc943f52" containerName="console"
Sep 10 14:46:03.666697 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.666510    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="105a530c-0f26-4d35-b8a3-bf39dc943f52" containerName="console"
Sep 10 14:46:03.668366 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.668350    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.672533 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.672513    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:46:03.673810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.673787    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-476tf\""
Sep 10 14:46:03.673916 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.673795    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:46:03.679178 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.679155    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"]
Sep 10 14:46:03.703609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.703584    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.703742 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.703615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.703742 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.703646    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87pxq\" (UniqueName: \"kubernetes.io/projected/0c95559e-ed08-4316-8b50-11b31701a255-kube-api-access-87pxq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.804606 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.804571    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.804606 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.804606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.804795 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.804637    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87pxq\" (UniqueName: \"kubernetes.io/projected/0c95559e-ed08-4316-8b50-11b31701a255-kube-api-access-87pxq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.804956 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.804936    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.804993 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.804969    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.812742 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.812710    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87pxq\" (UniqueName: \"kubernetes.io/projected/0c95559e-ed08-4316-8b50-11b31701a255-kube-api-access-87pxq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:03.977434 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:03.977402    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:04.246761 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:04.246736    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"]
Sep 10 14:46:04.251569 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:46:04.251530    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c95559e_ed08_4316_8b50_11b31701a255.slice/crio-341a6aaf19b8bb0776c213f461a3a73d1336133e956878394d58f27aea80b76d WatchSource:0}: Error finding container 341a6aaf19b8bb0776c213f461a3a73d1336133e956878394d58f27aea80b76d: Status 404 returned error can't find the container with id 341a6aaf19b8bb0776c213f461a3a73d1336133e956878394d58f27aea80b76d
Sep 10 14:46:05.086560 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:05.086529    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77" event={"ID":"0c95559e-ed08-4316-8b50-11b31701a255","Type":"ContainerStarted","Data":"341a6aaf19b8bb0776c213f461a3a73d1336133e956878394d58f27aea80b76d"}
Sep 10 14:46:14.124394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:14.124348    2558 generic.go:356] "Generic (PLEG): container finished" podID="0c95559e-ed08-4316-8b50-11b31701a255" containerID="0d13b64def9997e4cf29872c0b1018930875a5f12abc5285d78e0fe47f386d16" exitCode=0
Sep 10 14:46:14.124394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:14.124392    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77" event={"ID":"0c95559e-ed08-4316-8b50-11b31701a255","Type":"ContainerDied","Data":"0d13b64def9997e4cf29872c0b1018930875a5f12abc5285d78e0fe47f386d16"}
Sep 10 14:46:17.138307 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:17.138273    2558 generic.go:356] "Generic (PLEG): container finished" podID="0c95559e-ed08-4316-8b50-11b31701a255" containerID="a162ac593ba32ca8a94cfeac1be0fc6c1f8d9da0fb40cd532a0e7b39ec5c264b" exitCode=0
Sep 10 14:46:17.138700 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:17.138361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77" event={"ID":"0c95559e-ed08-4316-8b50-11b31701a255","Type":"ContainerDied","Data":"a162ac593ba32ca8a94cfeac1be0fc6c1f8d9da0fb40cd532a0e7b39ec5c264b"}
Sep 10 14:46:25.176984 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:25.176950    2558 generic.go:356] "Generic (PLEG): container finished" podID="0c95559e-ed08-4316-8b50-11b31701a255" containerID="4faf4cd5ede81d0dcccc7c31489e4a116522d90b55347f2b11156b2a704dff04" exitCode=0
Sep 10 14:46:25.177374 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:25.177019    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77" event={"ID":"0c95559e-ed08-4316-8b50-11b31701a255","Type":"ContainerDied","Data":"4faf4cd5ede81d0dcccc7c31489e4a116522d90b55347f2b11156b2a704dff04"}
Sep 10 14:46:26.317166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.317139    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:26.405499 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.405463    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87pxq\" (UniqueName: \"kubernetes.io/projected/0c95559e-ed08-4316-8b50-11b31701a255-kube-api-access-87pxq\") pod \"0c95559e-ed08-4316-8b50-11b31701a255\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") "
Sep 10 14:46:26.405668 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.405553    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-bundle\") pod \"0c95559e-ed08-4316-8b50-11b31701a255\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") "
Sep 10 14:46:26.405668 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.405573    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-util\") pod \"0c95559e-ed08-4316-8b50-11b31701a255\" (UID: \"0c95559e-ed08-4316-8b50-11b31701a255\") "
Sep 10 14:46:26.406187 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.406166    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-bundle" (OuterVolumeSpecName: "bundle") pod "0c95559e-ed08-4316-8b50-11b31701a255" (UID: "0c95559e-ed08-4316-8b50-11b31701a255"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:46:26.407811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.407796    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c95559e-ed08-4316-8b50-11b31701a255-kube-api-access-87pxq" (OuterVolumeSpecName: "kube-api-access-87pxq") pod "0c95559e-ed08-4316-8b50-11b31701a255" (UID: "0c95559e-ed08-4316-8b50-11b31701a255"). InnerVolumeSpecName "kube-api-access-87pxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:46:26.412862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.412835    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-util" (OuterVolumeSpecName: "util") pod "0c95559e-ed08-4316-8b50-11b31701a255" (UID: "0c95559e-ed08-4316-8b50-11b31701a255"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:46:26.506128 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.506082    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:26.506287 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.506134    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c95559e-ed08-4316-8b50-11b31701a255-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:26.506287 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:26.506152    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87pxq\" (UniqueName: \"kubernetes.io/projected/0c95559e-ed08-4316-8b50-11b31701a255-kube-api-access-87pxq\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:27.186793 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:27.186756    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77" event={"ID":"0c95559e-ed08-4316-8b50-11b31701a255","Type":"ContainerDied","Data":"341a6aaf19b8bb0776c213f461a3a73d1336133e956878394d58f27aea80b76d"}
Sep 10 14:46:27.186929 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:27.186801    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="341a6aaf19b8bb0776c213f461a3a73d1336133e956878394d58f27aea80b76d"
Sep 10 14:46:27.186929 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:27.186804    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vnv77"
Sep 10 14:46:31.681101 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681068    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"]
Sep 10 14:46:31.681593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681464    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c95559e-ed08-4316-8b50-11b31701a255" containerName="util"
Sep 10 14:46:31.681593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681478    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c95559e-ed08-4316-8b50-11b31701a255" containerName="util"
Sep 10 14:46:31.681593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681493    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c95559e-ed08-4316-8b50-11b31701a255" containerName="pull"
Sep 10 14:46:31.681593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681499    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c95559e-ed08-4316-8b50-11b31701a255" containerName="pull"
Sep 10 14:46:31.681593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681512    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c95559e-ed08-4316-8b50-11b31701a255" containerName="extract"
Sep 10 14:46:31.681593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681517    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c95559e-ed08-4316-8b50-11b31701a255" containerName="extract"
Sep 10 14:46:31.681593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.681568    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c95559e-ed08-4316-8b50-11b31701a255" containerName="extract"
Sep 10 14:46:31.685418 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.685400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:31.688522 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.688504    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 14:46:31.689020 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.688999    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8n99q\""
Sep 10 14:46:31.689081 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.689043    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:46:31.701278 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.701257    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"]
Sep 10 14:46:31.847708 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.847665    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp7mj\" (UniqueName: \"kubernetes.io/projected/31433424-ac1d-4888-8d7d-d85f3902b738-kube-api-access-rp7mj\") pod \"cert-manager-operator-controller-manager-55c444bf86-4dkzq\" (UID: \"31433424-ac1d-4888-8d7d-d85f3902b738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:31.847708 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.847713    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31433424-ac1d-4888-8d7d-d85f3902b738-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4dkzq\" (UID: \"31433424-ac1d-4888-8d7d-d85f3902b738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:31.948531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.948500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31433424-ac1d-4888-8d7d-d85f3902b738-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4dkzq\" (UID: \"31433424-ac1d-4888-8d7d-d85f3902b738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:31.948648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.948593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp7mj\" (UniqueName: \"kubernetes.io/projected/31433424-ac1d-4888-8d7d-d85f3902b738-kube-api-access-rp7mj\") pod \"cert-manager-operator-controller-manager-55c444bf86-4dkzq\" (UID: \"31433424-ac1d-4888-8d7d-d85f3902b738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:31.948961 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.948939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31433424-ac1d-4888-8d7d-d85f3902b738-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4dkzq\" (UID: \"31433424-ac1d-4888-8d7d-d85f3902b738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:31.957267 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.957245    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp7mj\" (UniqueName: \"kubernetes.io/projected/31433424-ac1d-4888-8d7d-d85f3902b738-kube-api-access-rp7mj\") pod \"cert-manager-operator-controller-manager-55c444bf86-4dkzq\" (UID: \"31433424-ac1d-4888-8d7d-d85f3902b738\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:31.995113 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:31.995084    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"
Sep 10 14:46:32.276289 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:32.276262    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq"]
Sep 10 14:46:32.277974 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:46:32.277950    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31433424_ac1d_4888_8d7d_d85f3902b738.slice/crio-055f024744e075d1938bbf2ee203ef611a4cab6adfc58c4b5d70759730617505 WatchSource:0}: Error finding container 055f024744e075d1938bbf2ee203ef611a4cab6adfc58c4b5d70759730617505: Status 404 returned error can't find the container with id 055f024744e075d1938bbf2ee203ef611a4cab6adfc58c4b5d70759730617505
Sep 10 14:46:33.218479 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:33.218394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq" event={"ID":"31433424-ac1d-4888-8d7d-d85f3902b738","Type":"ContainerStarted","Data":"055f024744e075d1938bbf2ee203ef611a4cab6adfc58c4b5d70759730617505"}
Sep 10 14:46:34.226426 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:34.226386    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq" event={"ID":"31433424-ac1d-4888-8d7d-d85f3902b738","Type":"ContainerStarted","Data":"ea149a72c9a84fb07d37c78693818b34883c3c9dcbc40981fdd1aca595f85e42"}
Sep 10 14:46:34.248567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:34.248517    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4dkzq" podStartSLOduration=1.448293245 podStartE2EDuration="3.248499376s" totalImagesPullingTime="1.800206131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:34.246842051 +0000 UTC m=+238.733529942" watchObservedRunningTime="2026-09-10 14:46:34.248499376 +0000 UTC m=+238.735187267"
Sep 10 14:46:35.147712 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.147684    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-px4mh"]
Sep 10 14:46:35.149881 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.149865    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.152491 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.152458    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 14:46:35.152625 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.152559    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 14:46:35.153428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.153413    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2zchs\""
Sep 10 14:46:35.160702 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.160678    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-px4mh"]
Sep 10 14:46:35.277024 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.276987    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fldvm\" (UniqueName: \"kubernetes.io/projected/f10a1a91-e20a-4c86-8b1c-56c6b8bca083-kube-api-access-fldvm\") pod \"cert-manager-webhook-6486ffd997-px4mh\" (UID: \"f10a1a91-e20a-4c86-8b1c-56c6b8bca083\") " pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.277420 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.277042    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a1a91-e20a-4c86-8b1c-56c6b8bca083-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-px4mh\" (UID: \"f10a1a91-e20a-4c86-8b1c-56c6b8bca083\") " pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.364348 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.364315    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"]
Sep 10 14:46:35.367155 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.367133    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.369834 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.369798    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:46:35.369957 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.369854    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:46:35.369957 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.369920    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-476tf\""
Sep 10 14:46:35.376289 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.376268    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"]
Sep 10 14:46:35.378429 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.378408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fldvm\" (UniqueName: \"kubernetes.io/projected/f10a1a91-e20a-4c86-8b1c-56c6b8bca083-kube-api-access-fldvm\") pod \"cert-manager-webhook-6486ffd997-px4mh\" (UID: \"f10a1a91-e20a-4c86-8b1c-56c6b8bca083\") " pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.378667 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.378639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a1a91-e20a-4c86-8b1c-56c6b8bca083-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-px4mh\" (UID: \"f10a1a91-e20a-4c86-8b1c-56c6b8bca083\") " pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.386679 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.386655    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f10a1a91-e20a-4c86-8b1c-56c6b8bca083-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-px4mh\" (UID: \"f10a1a91-e20a-4c86-8b1c-56c6b8bca083\") " pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.386897 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.386876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fldvm\" (UniqueName: \"kubernetes.io/projected/f10a1a91-e20a-4c86-8b1c-56c6b8bca083-kube-api-access-fldvm\") pod \"cert-manager-webhook-6486ffd997-px4mh\" (UID: \"f10a1a91-e20a-4c86-8b1c-56c6b8bca083\") " pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.469972 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.469940    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:35.479255 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.479201    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.479364 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.479297    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.479521 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.479366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5x62\" (UniqueName: \"kubernetes.io/projected/d562ce82-1304-4188-8f65-035837a68cd5-kube-api-access-h5x62\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.580896 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.580853    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5x62\" (UniqueName: \"kubernetes.io/projected/d562ce82-1304-4188-8f65-035837a68cd5-kube-api-access-h5x62\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.581075 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.580911    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.581075 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.580962    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.581296 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.581277    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.581352 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.581301    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.589451 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.589428    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5x62\" (UniqueName: \"kubernetes.io/projected/d562ce82-1304-4188-8f65-035837a68cd5-kube-api-access-h5x62\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.676819 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.676780    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:35.752944 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.752903    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-px4mh"]
Sep 10 14:46:35.761794 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:46:35.761763    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf10a1a91_e20a_4c86_8b1c_56c6b8bca083.slice/crio-be1fad0008ab85f01dd3ad2261469eb7743777c4bdd3b2a7d30376496beeba69 WatchSource:0}: Error finding container be1fad0008ab85f01dd3ad2261469eb7743777c4bdd3b2a7d30376496beeba69: Status 404 returned error can't find the container with id be1fad0008ab85f01dd3ad2261469eb7743777c4bdd3b2a7d30376496beeba69
Sep 10 14:46:35.947798 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:35.947770    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"]
Sep 10 14:46:35.950256 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:46:35.950213    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd562ce82_1304_4188_8f65_035837a68cd5.slice/crio-0167c919927f2764a0b0818d8c11071ac43f15a1562d8b4f5d757e84e6646130 WatchSource:0}: Error finding container 0167c919927f2764a0b0818d8c11071ac43f15a1562d8b4f5d757e84e6646130: Status 404 returned error can't find the container with id 0167c919927f2764a0b0818d8c11071ac43f15a1562d8b4f5d757e84e6646130
Sep 10 14:46:36.236342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:36.236252    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh" event={"ID":"f10a1a91-e20a-4c86-8b1c-56c6b8bca083","Type":"ContainerStarted","Data":"be1fad0008ab85f01dd3ad2261469eb7743777c4bdd3b2a7d30376496beeba69"}
Sep 10 14:46:36.237610 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:36.237583    2558 generic.go:356] "Generic (PLEG): container finished" podID="d562ce82-1304-4188-8f65-035837a68cd5" containerID="5cb6f760730260880c6cfab71815183ab394c348c4c7386dfa2c1758733605b6" exitCode=0
Sep 10 14:46:36.237726 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:36.237640    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd" event={"ID":"d562ce82-1304-4188-8f65-035837a68cd5","Type":"ContainerDied","Data":"5cb6f760730260880c6cfab71815183ab394c348c4c7386dfa2c1758733605b6"}
Sep 10 14:46:36.237726 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:36.237679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd" event={"ID":"d562ce82-1304-4188-8f65-035837a68cd5","Type":"ContainerStarted","Data":"0167c919927f2764a0b0818d8c11071ac43f15a1562d8b4f5d757e84e6646130"}
Sep 10 14:46:37.945984 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:37.945948    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"]
Sep 10 14:46:37.948873 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:37.948852    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:37.951267 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:37.951205    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9lv9w\""
Sep 10 14:46:37.956587 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:37.956535    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"]
Sep 10 14:46:38.105009 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:38.104976    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbnm6\" (UniqueName: \"kubernetes.io/projected/68c60b26-061a-4fee-ad40-19085b314940-kube-api-access-bbnm6\") pod \"cert-manager-cainjector-559cdd8c87-rqx9g\" (UID: \"68c60b26-061a-4fee-ad40-19085b314940\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:38.105188 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:38.105138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68c60b26-061a-4fee-ad40-19085b314940-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqx9g\" (UID: \"68c60b26-061a-4fee-ad40-19085b314940\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:38.206461 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:38.206373    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68c60b26-061a-4fee-ad40-19085b314940-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqx9g\" (UID: \"68c60b26-061a-4fee-ad40-19085b314940\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:38.206616 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:38.206467    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbnm6\" (UniqueName: \"kubernetes.io/projected/68c60b26-061a-4fee-ad40-19085b314940-kube-api-access-bbnm6\") pod \"cert-manager-cainjector-559cdd8c87-rqx9g\" (UID: \"68c60b26-061a-4fee-ad40-19085b314940\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:38.215284 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:38.215255    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/68c60b26-061a-4fee-ad40-19085b314940-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqx9g\" (UID: \"68c60b26-061a-4fee-ad40-19085b314940\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:38.215503 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:38.215477    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbnm6\" (UniqueName: \"kubernetes.io/projected/68c60b26-061a-4fee-ad40-19085b314940-kube-api-access-bbnm6\") pod \"cert-manager-cainjector-559cdd8c87-rqx9g\" (UID: \"68c60b26-061a-4fee-ad40-19085b314940\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:38.260735 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:38.260707    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"
Sep 10 14:46:39.254548 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:39.254491    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh" event={"ID":"f10a1a91-e20a-4c86-8b1c-56c6b8bca083","Type":"ContainerStarted","Data":"6b2673c3b26c9850abe9d7390e3b7fc27557e7adf77bda3c20bdd5b466149761"}
Sep 10 14:46:39.254951 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:39.254557    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:39.256294 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:39.256269    2558 generic.go:356] "Generic (PLEG): container finished" podID="d562ce82-1304-4188-8f65-035837a68cd5" containerID="6fcb49d8d9413039577380a3c1b591972a8690b8af29e10f9ae93589655d6e9d" exitCode=0
Sep 10 14:46:39.256431 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:39.256351    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd" event={"ID":"d562ce82-1304-4188-8f65-035837a68cd5","Type":"ContainerDied","Data":"6fcb49d8d9413039577380a3c1b591972a8690b8af29e10f9ae93589655d6e9d"}
Sep 10 14:46:39.272186 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:39.272134    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh" podStartSLOduration=0.974392015 podStartE2EDuration="4.272113453s" totalImagesPullingTime="3.297721438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:39.272069117 +0000 UTC m=+243.758757009" watchObservedRunningTime="2026-09-10 14:46:39.272113453 +0000 UTC m=+243.758801346"
Sep 10 14:46:39.311770 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:39.311742    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g"]
Sep 10 14:46:39.314512 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:46:39.314487    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68c60b26_061a_4fee_ad40_19085b314940.slice/crio-39072fdc19c4539c0f40cfd9b62f3dd2c871a1ca150d0a6d0f5f56ff5dc424e5 WatchSource:0}: Error finding container 39072fdc19c4539c0f40cfd9b62f3dd2c871a1ca150d0a6d0f5f56ff5dc424e5: Status 404 returned error can't find the container with id 39072fdc19c4539c0f40cfd9b62f3dd2c871a1ca150d0a6d0f5f56ff5dc424e5
Sep 10 14:46:40.263502 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:40.263471    2558 generic.go:356] "Generic (PLEG): container finished" podID="d562ce82-1304-4188-8f65-035837a68cd5" containerID="63dc2154d9aa24893c895c8931fcb72781ea8dbf327a9085941df67bb20e0dc8" exitCode=0
Sep 10 14:46:40.263949 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:40.263537    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd" event={"ID":"d562ce82-1304-4188-8f65-035837a68cd5","Type":"ContainerDied","Data":"63dc2154d9aa24893c895c8931fcb72781ea8dbf327a9085941df67bb20e0dc8"}
Sep 10 14:46:40.265258 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:40.265231    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g" event={"ID":"68c60b26-061a-4fee-ad40-19085b314940","Type":"ContainerStarted","Data":"55ae52358146143f0c506118ef49fbc295b39e7f900617050be4ecdea5e2c622"}
Sep 10 14:46:40.265393 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:40.265270    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g" event={"ID":"68c60b26-061a-4fee-ad40-19085b314940","Type":"ContainerStarted","Data":"39072fdc19c4539c0f40cfd9b62f3dd2c871a1ca150d0a6d0f5f56ff5dc424e5"}
Sep 10 14:46:40.303463 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:40.303415    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqx9g" podStartSLOduration=3.30339904 podStartE2EDuration="3.30339904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:40.302812263 +0000 UTC m=+244.789500175" watchObservedRunningTime="2026-09-10 14:46:40.30339904 +0000 UTC m=+244.790086959"
Sep 10 14:46:41.397153 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.397119    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:41.539211 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.539123    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5x62\" (UniqueName: \"kubernetes.io/projected/d562ce82-1304-4188-8f65-035837a68cd5-kube-api-access-h5x62\") pod \"d562ce82-1304-4188-8f65-035837a68cd5\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") "
Sep 10 14:46:41.539211 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.539169    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-bundle\") pod \"d562ce82-1304-4188-8f65-035837a68cd5\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") "
Sep 10 14:46:41.539211 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.539199    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-util\") pod \"d562ce82-1304-4188-8f65-035837a68cd5\" (UID: \"d562ce82-1304-4188-8f65-035837a68cd5\") "
Sep 10 14:46:41.539538 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.539498    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-bundle" (OuterVolumeSpecName: "bundle") pod "d562ce82-1304-4188-8f65-035837a68cd5" (UID: "d562ce82-1304-4188-8f65-035837a68cd5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:46:41.541423 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.541389    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d562ce82-1304-4188-8f65-035837a68cd5-kube-api-access-h5x62" (OuterVolumeSpecName: "kube-api-access-h5x62") pod "d562ce82-1304-4188-8f65-035837a68cd5" (UID: "d562ce82-1304-4188-8f65-035837a68cd5"). InnerVolumeSpecName "kube-api-access-h5x62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:46:41.603648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.603606    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-util" (OuterVolumeSpecName: "util") pod "d562ce82-1304-4188-8f65-035837a68cd5" (UID: "d562ce82-1304-4188-8f65-035837a68cd5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:46:41.639943 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.639905    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5x62\" (UniqueName: \"kubernetes.io/projected/d562ce82-1304-4188-8f65-035837a68cd5-kube-api-access-h5x62\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:41.639943 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.639942    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:41.640074 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:41.639953    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d562ce82-1304-4188-8f65-035837a68cd5-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:42.275859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:42.275773    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd" event={"ID":"d562ce82-1304-4188-8f65-035837a68cd5","Type":"ContainerDied","Data":"0167c919927f2764a0b0818d8c11071ac43f15a1562d8b4f5d757e84e6646130"}
Sep 10 14:46:42.275859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:42.275819    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0167c919927f2764a0b0818d8c11071ac43f15a1562d8b4f5d757e84e6646130"
Sep 10 14:46:42.275859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:42.275788    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft7whd"
Sep 10 14:46:45.268049 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:45.268020    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-px4mh"
Sep 10 14:46:47.906459 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.906428    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-89pt4"]
Sep 10 14:46:47.909135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.907259    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d562ce82-1304-4188-8f65-035837a68cd5" containerName="extract"
Sep 10 14:46:47.909135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.907300    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d562ce82-1304-4188-8f65-035837a68cd5" containerName="extract"
Sep 10 14:46:47.909135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.907322    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d562ce82-1304-4188-8f65-035837a68cd5" containerName="util"
Sep 10 14:46:47.909135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.907339    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d562ce82-1304-4188-8f65-035837a68cd5" containerName="util"
Sep 10 14:46:47.909135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.907381    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d562ce82-1304-4188-8f65-035837a68cd5" containerName="pull"
Sep 10 14:46:47.909135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.907390    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d562ce82-1304-4188-8f65-035837a68cd5" containerName="pull"
Sep 10 14:46:47.909135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.907617    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d562ce82-1304-4188-8f65-035837a68cd5" containerName="extract"
Sep 10 14:46:47.912264 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.912234    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:47.915246 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.915209    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zngw5\""
Sep 10 14:46:47.919626 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.919606    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-89pt4"]
Sep 10 14:46:47.995750 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.995710    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fb3dc0f-33ad-4578-9f59-28f93905d805-bound-sa-token\") pod \"cert-manager-55b4d954d5-89pt4\" (UID: \"4fb3dc0f-33ad-4578-9f59-28f93905d805\") " pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:47.995938 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:47.995764    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tct75\" (UniqueName: \"kubernetes.io/projected/4fb3dc0f-33ad-4578-9f59-28f93905d805-kube-api-access-tct75\") pod \"cert-manager-55b4d954d5-89pt4\" (UID: \"4fb3dc0f-33ad-4578-9f59-28f93905d805\") " pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:48.097048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:48.097020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fb3dc0f-33ad-4578-9f59-28f93905d805-bound-sa-token\") pod \"cert-manager-55b4d954d5-89pt4\" (UID: \"4fb3dc0f-33ad-4578-9f59-28f93905d805\") " pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:48.097201 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:48.097065    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tct75\" (UniqueName: \"kubernetes.io/projected/4fb3dc0f-33ad-4578-9f59-28f93905d805-kube-api-access-tct75\") pod \"cert-manager-55b4d954d5-89pt4\" (UID: \"4fb3dc0f-33ad-4578-9f59-28f93905d805\") " pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:48.107718 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:48.107696    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fb3dc0f-33ad-4578-9f59-28f93905d805-bound-sa-token\") pod \"cert-manager-55b4d954d5-89pt4\" (UID: \"4fb3dc0f-33ad-4578-9f59-28f93905d805\") " pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:48.107804 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:48.107756    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tct75\" (UniqueName: \"kubernetes.io/projected/4fb3dc0f-33ad-4578-9f59-28f93905d805-kube-api-access-tct75\") pod \"cert-manager-55b4d954d5-89pt4\" (UID: \"4fb3dc0f-33ad-4578-9f59-28f93905d805\") " pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:48.222137 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:48.222102    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-89pt4"
Sep 10 14:46:48.491926 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:48.491900    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-89pt4"]
Sep 10 14:46:48.494239 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:46:48.494189    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fb3dc0f_33ad_4578_9f59_28f93905d805.slice/crio-54a1da3168efe72a28f2d9ed2f5b0fc72786d8c40c66b5e4e321728c21665a92 WatchSource:0}: Error finding container 54a1da3168efe72a28f2d9ed2f5b0fc72786d8c40c66b5e4e321728c21665a92: Status 404 returned error can't find the container with id 54a1da3168efe72a28f2d9ed2f5b0fc72786d8c40c66b5e4e321728c21665a92
Sep 10 14:46:49.308999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:49.308961    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-89pt4" event={"ID":"4fb3dc0f-33ad-4578-9f59-28f93905d805","Type":"ContainerStarted","Data":"4632d57ce051f9db40d1650fc2216656a790b374a2a5668dcc629ec2c80516bd"}
Sep 10 14:46:49.308999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:49.309013    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-89pt4" event={"ID":"4fb3dc0f-33ad-4578-9f59-28f93905d805","Type":"ContainerStarted","Data":"54a1da3168efe72a28f2d9ed2f5b0fc72786d8c40c66b5e4e321728c21665a92"}
Sep 10 14:46:49.327640 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:49.327596    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-89pt4" podStartSLOduration=2.3275809020000002 podStartE2EDuration="2.327580902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:49.325880617 +0000 UTC m=+253.812568509" watchObservedRunningTime="2026-09-10 14:46:49.327580902 +0000 UTC m=+253.814268793"
Sep 10 14:46:52.310802 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.310762    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"]
Sep 10 14:46:52.314590 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.314566    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.317306 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.317281    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:46:52.317404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.317312    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:46:52.317404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.317312    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-476tf\""
Sep 10 14:46:52.323086 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.323061    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"]
Sep 10 14:46:52.333086 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.333060    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r2c2\" (UniqueName: \"kubernetes.io/projected/4ad8461f-c5e0-41a1-84cf-08d23da9f224-kube-api-access-5r2c2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.333210 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.333113    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.333210 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.333132    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.434512 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.434473    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r2c2\" (UniqueName: \"kubernetes.io/projected/4ad8461f-c5e0-41a1-84cf-08d23da9f224-kube-api-access-5r2c2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.434703 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.434541    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.434703 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.434570    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.434923 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.434901    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.434979 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.434928    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.444369 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.444339    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r2c2\" (UniqueName: \"kubernetes.io/projected/4ad8461f-c5e0-41a1-84cf-08d23da9f224-kube-api-access-5r2c2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.625630 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.625547    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:52.900189 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:52.900130    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"]
Sep 10 14:46:52.903150 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:46:52.903123    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad8461f_c5e0_41a1_84cf_08d23da9f224.slice/crio-8228980c76323dccedc4c7d665cb4ad8065e5e8187426085bf94b4052b416fab WatchSource:0}: Error finding container 8228980c76323dccedc4c7d665cb4ad8065e5e8187426085bf94b4052b416fab: Status 404 returned error can't find the container with id 8228980c76323dccedc4c7d665cb4ad8065e5e8187426085bf94b4052b416fab
Sep 10 14:46:53.334581 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:53.334546    2558 generic.go:356] "Generic (PLEG): container finished" podID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerID="ee4e941869d65c4762f442b19cf2f5ea78d7be22de7c6ad54237380d16a9a9ec" exitCode=0
Sep 10 14:46:53.334979 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:53.334632    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l" event={"ID":"4ad8461f-c5e0-41a1-84cf-08d23da9f224","Type":"ContainerDied","Data":"ee4e941869d65c4762f442b19cf2f5ea78d7be22de7c6ad54237380d16a9a9ec"}
Sep 10 14:46:53.334979 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:53.334711    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l" event={"ID":"4ad8461f-c5e0-41a1-84cf-08d23da9f224","Type":"ContainerStarted","Data":"8228980c76323dccedc4c7d665cb4ad8065e5e8187426085bf94b4052b416fab"}
Sep 10 14:46:54.341292 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:54.341177    2558 generic.go:356] "Generic (PLEG): container finished" podID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerID="34847f644e1fa2af3475af31dc4bb735e86329098f55a6aa93a33892a8ebe7b2" exitCode=0
Sep 10 14:46:54.341292 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:54.341266    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l" event={"ID":"4ad8461f-c5e0-41a1-84cf-08d23da9f224","Type":"ContainerDied","Data":"34847f644e1fa2af3475af31dc4bb735e86329098f55a6aa93a33892a8ebe7b2"}
Sep 10 14:46:55.347793 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:55.347763    2558 generic.go:356] "Generic (PLEG): container finished" podID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerID="d3177006291e9397404f9388a41cf911310ad1915bc96a1f5aa149055e13eba9" exitCode=0
Sep 10 14:46:55.348138 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:55.347847    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l" event={"ID":"4ad8461f-c5e0-41a1-84cf-08d23da9f224","Type":"ContainerDied","Data":"d3177006291e9397404f9388a41cf911310ad1915bc96a1f5aa149055e13eba9"}
Sep 10 14:46:56.480487 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.480447    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:56.569263 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.569206    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r2c2\" (UniqueName: \"kubernetes.io/projected/4ad8461f-c5e0-41a1-84cf-08d23da9f224-kube-api-access-5r2c2\") pod \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") "
Sep 10 14:46:56.569459 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.569289    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-bundle\") pod \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") "
Sep 10 14:46:56.569459 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.569385    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-util\") pod \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\" (UID: \"4ad8461f-c5e0-41a1-84cf-08d23da9f224\") "
Sep 10 14:46:56.570018 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.569990    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-bundle" (OuterVolumeSpecName: "bundle") pod "4ad8461f-c5e0-41a1-84cf-08d23da9f224" (UID: "4ad8461f-c5e0-41a1-84cf-08d23da9f224"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:46:56.571562 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.571538    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad8461f-c5e0-41a1-84cf-08d23da9f224-kube-api-access-5r2c2" (OuterVolumeSpecName: "kube-api-access-5r2c2") pod "4ad8461f-c5e0-41a1-84cf-08d23da9f224" (UID: "4ad8461f-c5e0-41a1-84cf-08d23da9f224"). InnerVolumeSpecName "kube-api-access-5r2c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:46:56.575208 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.575175    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-util" (OuterVolumeSpecName: "util") pod "4ad8461f-c5e0-41a1-84cf-08d23da9f224" (UID: "4ad8461f-c5e0-41a1-84cf-08d23da9f224"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:46:56.670611 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.670531    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:56.670611 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.670559    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r2c2\" (UniqueName: \"kubernetes.io/projected/4ad8461f-c5e0-41a1-84cf-08d23da9f224-kube-api-access-5r2c2\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:56.670611 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:56.670570    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ad8461f-c5e0-41a1-84cf-08d23da9f224-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:46:57.358905 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:57.358827    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l"
Sep 10 14:46:57.359061 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:57.358821    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp42l" event={"ID":"4ad8461f-c5e0-41a1-84cf-08d23da9f224","Type":"ContainerDied","Data":"8228980c76323dccedc4c7d665cb4ad8065e5e8187426085bf94b4052b416fab"}
Sep 10 14:46:57.359061 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:46:57.358949    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8228980c76323dccedc4c7d665cb4ad8065e5e8187426085bf94b4052b416fab"
Sep 10 14:47:03.522776 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.522742    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"]
Sep 10 14:47:03.523213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.523110    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerName="pull"
Sep 10 14:47:03.523213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.523121    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerName="pull"
Sep 10 14:47:03.523213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.523133    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerName="util"
Sep 10 14:47:03.523213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.523142    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerName="util"
Sep 10 14:47:03.523213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.523153    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerName="extract"
Sep 10 14:47:03.523213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.523160    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerName="extract"
Sep 10 14:47:03.523213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.523214    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad8461f-c5e0-41a1-84cf-08d23da9f224" containerName="extract"
Sep 10 14:47:03.525415 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.525400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.527970 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.527942    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:47:03.528100 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.527974    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-476tf\""
Sep 10 14:47:03.529128 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.529101    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:47:03.538647 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.538611    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"]
Sep 10 14:47:03.628588 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.628552    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.628763 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.628614    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.628763 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.628689    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22kxq\" (UniqueName: \"kubernetes.io/projected/cef5d461-e942-46ae-9a65-deb8d7534de5-kube-api-access-22kxq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.729531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.729484    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.729719 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.729577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.729719 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.729630    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22kxq\" (UniqueName: \"kubernetes.io/projected/cef5d461-e942-46ae-9a65-deb8d7534de5-kube-api-access-22kxq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.729945 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.729923    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.730001 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.729977    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.738927 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.738892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22kxq\" (UniqueName: \"kubernetes.io/projected/cef5d461-e942-46ae-9a65-deb8d7534de5-kube-api-access-22kxq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:03.836714 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:03.836635    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:04.107323 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:04.107289    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"]
Sep 10 14:47:04.109416 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:47:04.109390    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef5d461_e942_46ae_9a65_deb8d7534de5.slice/crio-d846c7a34e6ff5a609201498f9d8c2a5fefb408603ea68e434feed938497016c WatchSource:0}: Error finding container d846c7a34e6ff5a609201498f9d8c2a5fefb408603ea68e434feed938497016c: Status 404 returned error can't find the container with id d846c7a34e6ff5a609201498f9d8c2a5fefb408603ea68e434feed938497016c
Sep 10 14:47:04.392350 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:04.392254    2558 generic.go:356] "Generic (PLEG): container finished" podID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerID="044443586a68051d59faac800ba12421b920ebbf28638ec8fa78aa7dba375464" exitCode=0
Sep 10 14:47:04.392350 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:04.392303    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw" event={"ID":"cef5d461-e942-46ae-9a65-deb8d7534de5","Type":"ContainerDied","Data":"044443586a68051d59faac800ba12421b920ebbf28638ec8fa78aa7dba375464"}
Sep 10 14:47:04.392350 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:04.392343    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw" event={"ID":"cef5d461-e942-46ae-9a65-deb8d7534de5","Type":"ContainerStarted","Data":"d846c7a34e6ff5a609201498f9d8c2a5fefb408603ea68e434feed938497016c"}
Sep 10 14:47:05.209257 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.209211    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"]
Sep 10 14:47:05.213398 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.213372    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.216169 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.216144    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 14:47:05.216342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.216181    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 14:47:05.216538 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.216518    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7smlt\""
Sep 10 14:47:05.216905 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.216690    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 14:47:05.216905 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.216732    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 14:47:05.229085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.229052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"]
Sep 10 14:47:05.344664 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.344579    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a77a12a-da84-4e91-a891-a7b9805dd875-tmp\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.344664 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.344613    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a77a12a-da84-4e91-a891-a7b9805dd875-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.344853 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.344694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5wnz\" (UniqueName: \"kubernetes.io/projected/1a77a12a-da84-4e91-a891-a7b9805dd875-kube-api-access-t5wnz\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.344853 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.344729    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a77a12a-da84-4e91-a891-a7b9805dd875-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.344853 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.344748    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a77a12a-da84-4e91-a891-a7b9805dd875-manifests\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.398784 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.398750    2558 generic.go:356] "Generic (PLEG): container finished" podID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerID="3ea94b044e8feeba1173a1c7a7ed26919bf71b2bc4a71e4332a576ba8955b476" exitCode=0
Sep 10 14:47:05.398970 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.398795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw" event={"ID":"cef5d461-e942-46ae-9a65-deb8d7534de5","Type":"ContainerDied","Data":"3ea94b044e8feeba1173a1c7a7ed26919bf71b2bc4a71e4332a576ba8955b476"}
Sep 10 14:47:05.445479 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.445458    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5wnz\" (UniqueName: \"kubernetes.io/projected/1a77a12a-da84-4e91-a891-a7b9805dd875-kube-api-access-t5wnz\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.445597 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.445495    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a77a12a-da84-4e91-a891-a7b9805dd875-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.445597 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.445514    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a77a12a-da84-4e91-a891-a7b9805dd875-manifests\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.445597 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.445554    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a77a12a-da84-4e91-a891-a7b9805dd875-tmp\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.445597 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.445575    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a77a12a-da84-4e91-a891-a7b9805dd875-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.446023 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.445992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1a77a12a-da84-4e91-a891-a7b9805dd875-manifests\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.447768 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.447745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a77a12a-da84-4e91-a891-a7b9805dd875-tmp\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.448729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.448705    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a77a12a-da84-4e91-a891-a7b9805dd875-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.448806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.448705    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1a77a12a-da84-4e91-a891-a7b9805dd875-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.459749 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.459725    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5wnz\" (UniqueName: \"kubernetes.io/projected/1a77a12a-da84-4e91-a891-a7b9805dd875-kube-api-access-t5wnz\") pod \"opendatahub-operator-controller-manager-68f84f89d7-lrlkh\" (UID: \"1a77a12a-da84-4e91-a891-a7b9805dd875\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.524406 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.524368    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:05.807211 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:05.807185    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"]
Sep 10 14:47:05.809578 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:47:05.809551    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a77a12a_da84_4e91_a891_a7b9805dd875.slice/crio-f879e48bd5f0618ea582ac59d101c6d7a860ab7646afa35e50a5f92580329a0c WatchSource:0}: Error finding container f879e48bd5f0618ea582ac59d101c6d7a860ab7646afa35e50a5f92580329a0c: Status 404 returned error can't find the container with id f879e48bd5f0618ea582ac59d101c6d7a860ab7646afa35e50a5f92580329a0c
Sep 10 14:47:06.406652 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:06.406618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh" event={"ID":"1a77a12a-da84-4e91-a891-a7b9805dd875","Type":"ContainerStarted","Data":"f879e48bd5f0618ea582ac59d101c6d7a860ab7646afa35e50a5f92580329a0c"}
Sep 10 14:47:06.409842 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:06.409456    2558 generic.go:356] "Generic (PLEG): container finished" podID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerID="eca36f8274aead65645fa4bb08869ab76516dce307db9388e04852a2a4ac91d1" exitCode=0
Sep 10 14:47:06.409842 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:06.409507    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw" event={"ID":"cef5d461-e942-46ae-9a65-deb8d7534de5","Type":"ContainerDied","Data":"eca36f8274aead65645fa4bb08869ab76516dce307db9388e04852a2a4ac91d1"}
Sep 10 14:47:08.961726 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:08.961702    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:09.075587 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.075563    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22kxq\" (UniqueName: \"kubernetes.io/projected/cef5d461-e942-46ae-9a65-deb8d7534de5-kube-api-access-22kxq\") pod \"cef5d461-e942-46ae-9a65-deb8d7534de5\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") "
Sep 10 14:47:09.075696 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.075621    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-bundle\") pod \"cef5d461-e942-46ae-9a65-deb8d7534de5\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") "
Sep 10 14:47:09.075696 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.075659    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-util\") pod \"cef5d461-e942-46ae-9a65-deb8d7534de5\" (UID: \"cef5d461-e942-46ae-9a65-deb8d7534de5\") "
Sep 10 14:47:09.076297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.076266    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-bundle" (OuterVolumeSpecName: "bundle") pod "cef5d461-e942-46ae-9a65-deb8d7534de5" (UID: "cef5d461-e942-46ae-9a65-deb8d7534de5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:47:09.077800 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.077777    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef5d461-e942-46ae-9a65-deb8d7534de5-kube-api-access-22kxq" (OuterVolumeSpecName: "kube-api-access-22kxq") pod "cef5d461-e942-46ae-9a65-deb8d7534de5" (UID: "cef5d461-e942-46ae-9a65-deb8d7534de5"). InnerVolumeSpecName "kube-api-access-22kxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:47:09.082242 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.082098    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-util" (OuterVolumeSpecName: "util") pod "cef5d461-e942-46ae-9a65-deb8d7534de5" (UID: "cef5d461-e942-46ae-9a65-deb8d7534de5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:47:09.177111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.177078    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:09.177111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.177105    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cef5d461-e942-46ae-9a65-deb8d7534de5-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:09.177111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.177115    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22kxq\" (UniqueName: \"kubernetes.io/projected/cef5d461-e942-46ae-9a65-deb8d7534de5-kube-api-access-22kxq\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:47:09.428210 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.428116    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh" event={"ID":"1a77a12a-da84-4e91-a891-a7b9805dd875","Type":"ContainerStarted","Data":"622a340a2f7c5ed85bd9643c8415cf067d9f8965a1ff80240b18e040b9ac6fb6"}
Sep 10 14:47:09.430065 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.430037    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw" event={"ID":"cef5d461-e942-46ae-9a65-deb8d7534de5","Type":"ContainerDied","Data":"d846c7a34e6ff5a609201498f9d8c2a5fefb408603ea68e434feed938497016c"}
Sep 10 14:47:09.430164 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.430085    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d846c7a34e6ff5a609201498f9d8c2a5fefb408603ea68e434feed938497016c"
Sep 10 14:47:09.430164 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:09.430050    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0p8dsw"
Sep 10 14:47:10.435788 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:10.435708    2558 generic.go:356] "Generic (PLEG): container finished" podID="1a77a12a-da84-4e91-a891-a7b9805dd875" containerID="622a340a2f7c5ed85bd9643c8415cf067d9f8965a1ff80240b18e040b9ac6fb6" exitCode=0
Sep 10 14:47:10.436121 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:10.435795    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh" event={"ID":"1a77a12a-da84-4e91-a891-a7b9805dd875","Type":"ContainerDied","Data":"622a340a2f7c5ed85bd9643c8415cf067d9f8965a1ff80240b18e040b9ac6fb6"}
Sep 10 14:47:11.447843 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:11.447803    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh" event={"ID":"1a77a12a-da84-4e91-a891-a7b9805dd875","Type":"ContainerStarted","Data":"3cbcfe5cda715c075be5a95e169099ceb658153a7ccb89a1050805017138bdf1"}
Sep 10 14:47:11.448202 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:11.447893    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:11.471274 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:11.471199    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh" podStartSLOduration=1.69720782 podStartE2EDuration="6.471186107s" totalImagesPullingTime="3.75136537s" totalInitContainerRuntime="1.022612917s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:11.469839296 +0000 UTC m=+275.956527198" watchObservedRunningTime="2026-09-10 14:47:11.471186107 +0000 UTC m=+275.957873998"
Sep 10 14:47:22.455745 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:22.455712    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-lrlkh"
Sep 10 14:47:23.426602 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426569    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"]
Sep 10 14:47:23.426904 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426892    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerName="util"
Sep 10 14:47:23.426947 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426907    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerName="util"
Sep 10 14:47:23.426947 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426928    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerName="pull"
Sep 10 14:47:23.426947 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426933    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerName="pull"
Sep 10 14:47:23.426947 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426941    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerName="extract"
Sep 10 14:47:23.426947 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426947    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerName="extract"
Sep 10 14:47:23.427100 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.426993    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef5d461-e942-46ae-9a65-deb8d7534de5" containerName="extract"
Sep 10 14:47:23.434950 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.434926    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.438531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.438336    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:47:23.438531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.438351    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 14:47:23.438531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.438355    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8jf4z\""
Sep 10 14:47:23.438531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.438404    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 14:47:23.438531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.438356    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 14:47:23.439213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.439195    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 14:47:23.439695 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.439674    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"]
Sep 10 14:47:23.501470 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.501440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/83dd1085-7481-4cd4-9762-09e3758faa04-metrics-cert\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.501909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.501483    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd52z\" (UniqueName: \"kubernetes.io/projected/83dd1085-7481-4cd4-9762-09e3758faa04-kube-api-access-wd52z\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.501909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.501513    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/83dd1085-7481-4cd4-9762-09e3758faa04-manager-config\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.501909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.501668    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83dd1085-7481-4cd4-9762-09e3758faa04-cert\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.602860 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.602819    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/83dd1085-7481-4cd4-9762-09e3758faa04-metrics-cert\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.603051 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.602868    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd52z\" (UniqueName: \"kubernetes.io/projected/83dd1085-7481-4cd4-9762-09e3758faa04-kube-api-access-wd52z\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.603051 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.602899    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/83dd1085-7481-4cd4-9762-09e3758faa04-manager-config\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.603051 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.602965    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83dd1085-7481-4cd4-9762-09e3758faa04-cert\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.603678 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.603655    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/83dd1085-7481-4cd4-9762-09e3758faa04-manager-config\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.605692 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.605659    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/83dd1085-7481-4cd4-9762-09e3758faa04-metrics-cert\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.605692 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.605676    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83dd1085-7481-4cd4-9762-09e3758faa04-cert\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.611613 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.611589    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd52z\" (UniqueName: \"kubernetes.io/projected/83dd1085-7481-4cd4-9762-09e3758faa04-kube-api-access-wd52z\") pod \"lws-controller-manager-bc768b6-qm2tn\" (UID: \"83dd1085-7481-4cd4-9762-09e3758faa04\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:23.746571 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:23.746535    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:24.019174 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:24.019100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"]
Sep 10 14:47:24.022926 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:47:24.022901    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83dd1085_7481_4cd4_9762_09e3758faa04.slice/crio-2bc5afdce0cefa3060c3f0d44a989e32ea24c01e962a52899bb2ccd78f0fc862 WatchSource:0}: Error finding container 2bc5afdce0cefa3060c3f0d44a989e32ea24c01e962a52899bb2ccd78f0fc862: Status 404 returned error can't find the container with id 2bc5afdce0cefa3060c3f0d44a989e32ea24c01e962a52899bb2ccd78f0fc862
Sep 10 14:47:24.515878 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:24.515845    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn" event={"ID":"83dd1085-7481-4cd4-9762-09e3758faa04","Type":"ContainerStarted","Data":"2bc5afdce0cefa3060c3f0d44a989e32ea24c01e962a52899bb2ccd78f0fc862"}
Sep 10 14:47:25.460776 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.460742    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"]
Sep 10 14:47:25.464785 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.464759    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.469094 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.469068    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 14:47:25.469462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.469363    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 14:47:25.469462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.469398    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pcf7f\""
Sep 10 14:47:25.469462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.469406    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 14:47:25.469690 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.469532    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 14:47:25.481125 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.481092    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"]
Sep 10 14:47:25.623967 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.623926    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d187a2fb-7de0-4471-b2b1-77e8e936ba89-tmp\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.624419 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.624052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d187a2fb-7de0-4471-b2b1-77e8e936ba89-tls-certs\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.624419 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.624129    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t2dk\" (UniqueName: \"kubernetes.io/projected/d187a2fb-7de0-4471-b2b1-77e8e936ba89-kube-api-access-4t2dk\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.725521 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.725441    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d187a2fb-7de0-4471-b2b1-77e8e936ba89-tmp\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.725685 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.725530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d187a2fb-7de0-4471-b2b1-77e8e936ba89-tls-certs\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.725685 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.725582    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t2dk\" (UniqueName: \"kubernetes.io/projected/d187a2fb-7de0-4471-b2b1-77e8e936ba89-kube-api-access-4t2dk\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.727782 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.727746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d187a2fb-7de0-4471-b2b1-77e8e936ba89-tmp\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.728013 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.727993    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d187a2fb-7de0-4471-b2b1-77e8e936ba89-tls-certs\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.737498 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.737474    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t2dk\" (UniqueName: \"kubernetes.io/projected/d187a2fb-7de0-4471-b2b1-77e8e936ba89-kube-api-access-4t2dk\") pod \"kube-auth-proxy-79b7fd47cd-cgvr7\" (UID: \"d187a2fb-7de0-4471-b2b1-77e8e936ba89\") " pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:25.778625 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:25.778600    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"
Sep 10 14:47:26.060197 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:26.060173    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7"]
Sep 10 14:47:26.061822 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:47:26.061799    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd187a2fb_7de0_4471_b2b1_77e8e936ba89.slice/crio-4f9fa68b5510dd9f55f79c6453d92e77550d4bca6e768392662b39b8968d97f6 WatchSource:0}: Error finding container 4f9fa68b5510dd9f55f79c6453d92e77550d4bca6e768392662b39b8968d97f6: Status 404 returned error can't find the container with id 4f9fa68b5510dd9f55f79c6453d92e77550d4bca6e768392662b39b8968d97f6
Sep 10 14:47:26.527868 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:26.527835    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7" event={"ID":"d187a2fb-7de0-4471-b2b1-77e8e936ba89","Type":"ContainerStarted","Data":"4f9fa68b5510dd9f55f79c6453d92e77550d4bca6e768392662b39b8968d97f6"}
Sep 10 14:47:26.529302 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:26.529279    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn" event={"ID":"83dd1085-7481-4cd4-9762-09e3758faa04","Type":"ContainerStarted","Data":"9d0a60d06d82a5a59b7c391435e8b17617e5f76037f4fb1135d449dafd7324c9"}
Sep 10 14:47:26.529416 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:26.529403    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:47:26.566202 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:26.566147    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn" podStartSLOduration=1.8406558039999998 podStartE2EDuration="3.566131982s" totalImagesPullingTime="1.725476178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:26.565317383 +0000 UTC m=+291.052005275" watchObservedRunningTime="2026-09-10 14:47:26.566131982 +0000 UTC m=+291.052819872"
Sep 10 14:47:30.553723 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:30.553638    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7" event={"ID":"d187a2fb-7de0-4471-b2b1-77e8e936ba89","Type":"ContainerStarted","Data":"50da6e58718198fceed670fcf55fedfa3657ee1016f872dc524140163d089098"}
Sep 10 14:47:30.572870 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:30.572806    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79b7fd47cd-cgvr7" podStartSLOduration=1.410993229 podStartE2EDuration="5.572787856s" totalImagesPullingTime="4.161794627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:30.572040805 +0000 UTC m=+295.058728695" watchObservedRunningTime="2026-09-10 14:47:30.572787856 +0000 UTC m=+295.059475746"
Sep 10 14:47:35.904265 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:35.904240    2558 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 14:47:37.537717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:47:37.537689    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bc768b6-qm2tn"
Sep 10 14:48:44.208783 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.208699    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"]
Sep 10 14:48:44.211022 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.211005    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.213753 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.213731    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-476tf\""
Sep 10 14:48:44.214439 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.214423    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 14:48:44.214763 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.214748    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 14:48:44.229534 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.229503    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"]
Sep 10 14:48:44.292136 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.292099    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"]
Sep 10 14:48:44.294492 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.294475    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.309428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.309388    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-974cg\" (UniqueName: \"kubernetes.io/projected/1b0894a5-3327-4d8e-83a8-948993ddfe30-kube-api-access-974cg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.309565 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.309476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.309565 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.309511    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.315685 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.315659    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"]
Sep 10 14:48:44.410572 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.410535    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.410572 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.410569    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxc2w\" (UniqueName: \"kubernetes.io/projected/6fba1b84-197e-43a8-b37b-b52f630d58f8-kube-api-access-fxc2w\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.410797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.410595    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974cg\" (UniqueName: \"kubernetes.io/projected/1b0894a5-3327-4d8e-83a8-948993ddfe30-kube-api-access-974cg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.410797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.410673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.410797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.410712    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.410797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.410731    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.411002 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.410987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.411107 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.411088    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.418617 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.418590    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"]
Sep 10 14:48:44.420983 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.420966    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.430960 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.430936    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-974cg\" (UniqueName: \"kubernetes.io/projected/1b0894a5-3327-4d8e-83a8-948993ddfe30-kube-api-access-974cg\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.438749 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.438720    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"]
Sep 10 14:48:44.511986 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.511897    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47tw2\" (UniqueName: \"kubernetes.io/projected/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-kube-api-access-47tw2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.511986 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.511955    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.512170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.511998    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxc2w\" (UniqueName: \"kubernetes.io/projected/6fba1b84-197e-43a8-b37b-b52f630d58f8-kube-api-access-fxc2w\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.512170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.512049    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.512170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.512080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.512170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.512140    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.512404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.512317    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.512551 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.512531    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.520814 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.520799    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:44.528334 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.528310    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"]
Sep 10 14:48:44.529989 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.529958    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxc2w\" (UniqueName: \"kubernetes.io/projected/6fba1b84-197e-43a8-b37b-b52f630d58f8-kube-api-access-fxc2w\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.530841 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.530818    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.552371 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.552341    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"]
Sep 10 14:48:44.603048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.603018    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:44.613324 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613299    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.613404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613339    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.613404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613381    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbnjd\" (UniqueName: \"kubernetes.io/projected/13f96842-7202-4732-b60a-ff97323f64d3-kube-api-access-zbnjd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.613559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.613559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613475    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47tw2\" (UniqueName: \"kubernetes.io/projected/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-kube-api-access-47tw2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.613559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613500    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.613830 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.613830 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.613748    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.623895 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.623866    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47tw2\" (UniqueName: \"kubernetes.io/projected/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-kube-api-access-47tw2\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.714623 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.714586    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnjd\" (UniqueName: \"kubernetes.io/projected/13f96842-7202-4732-b60a-ff97323f64d3-kube-api-access-zbnjd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.714793 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.714642    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.714793 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.714665    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.715037 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.715022    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.715073 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.715045    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.724394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.724364    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbnjd\" (UniqueName: \"kubernetes.io/projected/13f96842-7202-4732-b60a-ff97323f64d3-kube-api-access-zbnjd\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.730172 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.730152    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:44.804332 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.804246    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"]
Sep 10 14:48:44.806094 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:48:44.806066    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b0894a5_3327_4d8e_83a8_948993ddfe30.slice/crio-d529b2fa48166191280fda8070ef6356d7d481a3655d81121c69b52f80ad5ee6 WatchSource:0}: Error finding container d529b2fa48166191280fda8070ef6356d7d481a3655d81121c69b52f80ad5ee6: Status 404 returned error can't find the container with id d529b2fa48166191280fda8070ef6356d7d481a3655d81121c69b52f80ad5ee6
Sep 10 14:48:44.852946 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.852919    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:44.883281 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.883171    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"]
Sep 10 14:48:44.885508 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:48:44.885481    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fba1b84_197e_43a8_b37b_b52f630d58f8.slice/crio-f1b9eb3532c5c30f4cbcadefa3a5dd2ba0bc92836d70849176bd2ce851348c27 WatchSource:0}: Error finding container f1b9eb3532c5c30f4cbcadefa3a5dd2ba0bc92836d70849176bd2ce851348c27: Status 404 returned error can't find the container with id f1b9eb3532c5c30f4cbcadefa3a5dd2ba0bc92836d70849176bd2ce851348c27
Sep 10 14:48:44.923798 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.923622    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff" event={"ID":"1b0894a5-3327-4d8e-83a8-948993ddfe30","Type":"ContainerStarted","Data":"89a43cc5185a9ed6c1da4ad189793bb58eb48cdb08c376707729ed8e4fa3c084"}
Sep 10 14:48:44.923798 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.923713    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff" event={"ID":"1b0894a5-3327-4d8e-83a8-948993ddfe30","Type":"ContainerStarted","Data":"d529b2fa48166191280fda8070ef6356d7d481a3655d81121c69b52f80ad5ee6"}
Sep 10 14:48:44.926099 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:44.926072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b" event={"ID":"6fba1b84-197e-43a8-b37b-b52f630d58f8","Type":"ContainerStarted","Data":"f1b9eb3532c5c30f4cbcadefa3a5dd2ba0bc92836d70849176bd2ce851348c27"}
Sep 10 14:48:45.018311 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.018286    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"]
Sep 10 14:48:45.020620 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:48:45.020594    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5dcdb4_d4a5_44ca_a735_44521157a2ca.slice/crio-ef67f064cb477eaab28ce3fbaa511d7dc6d3b70fea77df8eb370a09e84831a8f WatchSource:0}: Error finding container ef67f064cb477eaab28ce3fbaa511d7dc6d3b70fea77df8eb370a09e84831a8f: Status 404 returned error can't find the container with id ef67f064cb477eaab28ce3fbaa511d7dc6d3b70fea77df8eb370a09e84831a8f
Sep 10 14:48:45.147701 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.147665    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"]
Sep 10 14:48:45.148744 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:48:45.148707    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f96842_7202_4732_b60a_ff97323f64d3.slice/crio-5faca1e8dc1fd04dd10a2584e85785043f375e024691aaf655e596f4716e184e WatchSource:0}: Error finding container 5faca1e8dc1fd04dd10a2584e85785043f375e024691aaf655e596f4716e184e: Status 404 returned error can't find the container with id 5faca1e8dc1fd04dd10a2584e85785043f375e024691aaf655e596f4716e184e
Sep 10 14:48:45.932344 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.932307    2558 generic.go:356] "Generic (PLEG): container finished" podID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerID="89a43cc5185a9ed6c1da4ad189793bb58eb48cdb08c376707729ed8e4fa3c084" exitCode=0
Sep 10 14:48:45.932760 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.932391    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff" event={"ID":"1b0894a5-3327-4d8e-83a8-948993ddfe30","Type":"ContainerDied","Data":"89a43cc5185a9ed6c1da4ad189793bb58eb48cdb08c376707729ed8e4fa3c084"}
Sep 10 14:48:45.933488 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.933470    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:48:45.934037 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.934015    2558 generic.go:356] "Generic (PLEG): container finished" podID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerID="3266e408cb340a7cc00f820f39b848c87f36b07f03bf67fdb1e98ada4a689c33" exitCode=0
Sep 10 14:48:45.934134 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.934109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn" event={"ID":"4e5dcdb4-d4a5-44ca-a735-44521157a2ca","Type":"ContainerDied","Data":"3266e408cb340a7cc00f820f39b848c87f36b07f03bf67fdb1e98ada4a689c33"}
Sep 10 14:48:45.934211 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.934166    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn" event={"ID":"4e5dcdb4-d4a5-44ca-a735-44521157a2ca","Type":"ContainerStarted","Data":"ef67f064cb477eaab28ce3fbaa511d7dc6d3b70fea77df8eb370a09e84831a8f"}
Sep 10 14:48:45.935671 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.935652    2558 generic.go:356] "Generic (PLEG): container finished" podID="13f96842-7202-4732-b60a-ff97323f64d3" containerID="be644a834d048876c1eb145221aee23a2fe8f2c01e250a99588ff5304f110426" exitCode=0
Sep 10 14:48:45.935737 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.935703    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl" event={"ID":"13f96842-7202-4732-b60a-ff97323f64d3","Type":"ContainerDied","Data":"be644a834d048876c1eb145221aee23a2fe8f2c01e250a99588ff5304f110426"}
Sep 10 14:48:45.935785 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.935740    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl" event={"ID":"13f96842-7202-4732-b60a-ff97323f64d3","Type":"ContainerStarted","Data":"5faca1e8dc1fd04dd10a2584e85785043f375e024691aaf655e596f4716e184e"}
Sep 10 14:48:45.937483 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.937459    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerID="6050151c5c57c7611579fa9d61ce9ad91a11a575606facf21c6e6d255d35dbe6" exitCode=0
Sep 10 14:48:45.937574 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:45.937536    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b" event={"ID":"6fba1b84-197e-43a8-b37b-b52f630d58f8","Type":"ContainerDied","Data":"6050151c5c57c7611579fa9d61ce9ad91a11a575606facf21c6e6d255d35dbe6"}
Sep 10 14:48:47.950889 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:47.950850    2558 generic.go:356] "Generic (PLEG): container finished" podID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerID="c8517c76123ad50de44466896d1b8660d57145ec52574d2b84b1a75575448bb8" exitCode=0
Sep 10 14:48:47.951310 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:47.950930    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff" event={"ID":"1b0894a5-3327-4d8e-83a8-948993ddfe30","Type":"ContainerDied","Data":"c8517c76123ad50de44466896d1b8660d57145ec52574d2b84b1a75575448bb8"}
Sep 10 14:48:47.952945 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:47.952919    2558 generic.go:356] "Generic (PLEG): container finished" podID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerID="aeaf71cb73c17b2ea528c073ec34b228b302b3c4f718aba2646c31ff088bc9e7" exitCode=0
Sep 10 14:48:47.953066 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:47.952998    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn" event={"ID":"4e5dcdb4-d4a5-44ca-a735-44521157a2ca","Type":"ContainerDied","Data":"aeaf71cb73c17b2ea528c073ec34b228b302b3c4f718aba2646c31ff088bc9e7"}
Sep 10 14:48:47.955000 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:47.954979    2558 generic.go:356] "Generic (PLEG): container finished" podID="13f96842-7202-4732-b60a-ff97323f64d3" containerID="ba1ffe957c01c9996c9c7149692b652bc48d5e23e043c2f7998466a77ac18894" exitCode=0
Sep 10 14:48:47.955090 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:47.955052    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl" event={"ID":"13f96842-7202-4732-b60a-ff97323f64d3","Type":"ContainerDied","Data":"ba1ffe957c01c9996c9c7149692b652bc48d5e23e043c2f7998466a77ac18894"}
Sep 10 14:48:48.961812 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:48.961779    2558 generic.go:356] "Generic (PLEG): container finished" podID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerID="1f9e4e9a0134ef52b935e66f1593478ecb46769044d347a2a41916b1f09b1866" exitCode=0
Sep 10 14:48:48.962304 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:48.961863    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff" event={"ID":"1b0894a5-3327-4d8e-83a8-948993ddfe30","Type":"ContainerDied","Data":"1f9e4e9a0134ef52b935e66f1593478ecb46769044d347a2a41916b1f09b1866"}
Sep 10 14:48:48.963960 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:48.963937    2558 generic.go:356] "Generic (PLEG): container finished" podID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerID="c5b902513ae1df6a987f64c8f9aab16c093cdabe98b67dbbbc7b442d6de594f0" exitCode=0
Sep 10 14:48:48.964045 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:48.964024    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn" event={"ID":"4e5dcdb4-d4a5-44ca-a735-44521157a2ca","Type":"ContainerDied","Data":"c5b902513ae1df6a987f64c8f9aab16c093cdabe98b67dbbbc7b442d6de594f0"}
Sep 10 14:48:48.965903 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:48.965882    2558 generic.go:356] "Generic (PLEG): container finished" podID="13f96842-7202-4732-b60a-ff97323f64d3" containerID="ca4cf581ea6664e07d4fc8cedf79eb91f3322e4e3a5d9a8715e1a3acdf3c9db0" exitCode=0
Sep 10 14:48:48.966001 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:48.965961    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl" event={"ID":"13f96842-7202-4732-b60a-ff97323f64d3","Type":"ContainerDied","Data":"ca4cf581ea6664e07d4fc8cedf79eb91f3322e4e3a5d9a8715e1a3acdf3c9db0"}
Sep 10 14:48:49.973310 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:49.973274    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerID="b0a19b6b9ee861364049f2674eba809941b1e2bace1cfdde5992f360de562f6b" exitCode=0
Sep 10 14:48:49.973736 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:49.973349    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b" event={"ID":"6fba1b84-197e-43a8-b37b-b52f630d58f8","Type":"ContainerDied","Data":"b0a19b6b9ee861364049f2674eba809941b1e2bace1cfdde5992f360de562f6b"}
Sep 10 14:48:50.121311 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.121288    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:50.165520 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.165491    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-bundle\") pod \"1b0894a5-3327-4d8e-83a8-948993ddfe30\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") "
Sep 10 14:48:50.165706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.165558    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-util\") pod \"1b0894a5-3327-4d8e-83a8-948993ddfe30\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") "
Sep 10 14:48:50.165706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.165604    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-974cg\" (UniqueName: \"kubernetes.io/projected/1b0894a5-3327-4d8e-83a8-948993ddfe30-kube-api-access-974cg\") pod \"1b0894a5-3327-4d8e-83a8-948993ddfe30\" (UID: \"1b0894a5-3327-4d8e-83a8-948993ddfe30\") "
Sep 10 14:48:50.165929 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.165900    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-bundle" (OuterVolumeSpecName: "bundle") pod "1b0894a5-3327-4d8e-83a8-948993ddfe30" (UID: "1b0894a5-3327-4d8e-83a8-948993ddfe30"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:50.167903 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.167877    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b0894a5-3327-4d8e-83a8-948993ddfe30-kube-api-access-974cg" (OuterVolumeSpecName: "kube-api-access-974cg") pod "1b0894a5-3327-4d8e-83a8-948993ddfe30" (UID: "1b0894a5-3327-4d8e-83a8-948993ddfe30"). InnerVolumeSpecName "kube-api-access-974cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:50.193607 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.193584    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:50.197203 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.197186    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:50.257241 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.257134    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-util" (OuterVolumeSpecName: "util") pod "1b0894a5-3327-4d8e-83a8-948993ddfe30" (UID: "1b0894a5-3327-4d8e-83a8-948993ddfe30"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:50.266178 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266148    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-bundle\") pod \"13f96842-7202-4732-b60a-ff97323f64d3\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") "
Sep 10 14:48:50.266346 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266189    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-util\") pod \"13f96842-7202-4732-b60a-ff97323f64d3\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") "
Sep 10 14:48:50.266346 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266249    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbnjd\" (UniqueName: \"kubernetes.io/projected/13f96842-7202-4732-b60a-ff97323f64d3-kube-api-access-zbnjd\") pod \"13f96842-7202-4732-b60a-ff97323f64d3\" (UID: \"13f96842-7202-4732-b60a-ff97323f64d3\") "
Sep 10 14:48:50.266346 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266289    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47tw2\" (UniqueName: \"kubernetes.io/projected/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-kube-api-access-47tw2\") pod \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") "
Sep 10 14:48:50.266346 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266324    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-util\") pod \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") "
Sep 10 14:48:50.266549 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266397    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-bundle\") pod \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\" (UID: \"4e5dcdb4-d4a5-44ca-a735-44521157a2ca\") "
Sep 10 14:48:50.266708 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266687    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.266772 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266715    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-974cg\" (UniqueName: \"kubernetes.io/projected/1b0894a5-3327-4d8e-83a8-948993ddfe30-kube-api-access-974cg\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.266772 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266729    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b0894a5-3327-4d8e-83a8-948993ddfe30-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.266963 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.266941    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-bundle" (OuterVolumeSpecName: "bundle") pod "13f96842-7202-4732-b60a-ff97323f64d3" (UID: "13f96842-7202-4732-b60a-ff97323f64d3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:50.267398 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.267166    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-bundle" (OuterVolumeSpecName: "bundle") pod "4e5dcdb4-d4a5-44ca-a735-44521157a2ca" (UID: "4e5dcdb4-d4a5-44ca-a735-44521157a2ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:50.268872 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.268846    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13f96842-7202-4732-b60a-ff97323f64d3-kube-api-access-zbnjd" (OuterVolumeSpecName: "kube-api-access-zbnjd") pod "13f96842-7202-4732-b60a-ff97323f64d3" (UID: "13f96842-7202-4732-b60a-ff97323f64d3"). InnerVolumeSpecName "kube-api-access-zbnjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:50.268966 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.268913    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-kube-api-access-47tw2" (OuterVolumeSpecName: "kube-api-access-47tw2") pod "4e5dcdb4-d4a5-44ca-a735-44521157a2ca" (UID: "4e5dcdb4-d4a5-44ca-a735-44521157a2ca"). InnerVolumeSpecName "kube-api-access-47tw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:50.272341 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.272315    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-util" (OuterVolumeSpecName: "util") pod "13f96842-7202-4732-b60a-ff97323f64d3" (UID: "13f96842-7202-4732-b60a-ff97323f64d3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:50.273341 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.273315    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-util" (OuterVolumeSpecName: "util") pod "4e5dcdb4-d4a5-44ca-a735-44521157a2ca" (UID: "4e5dcdb4-d4a5-44ca-a735-44521157a2ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:50.367931 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.367882    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.367931 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.367927    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbnjd\" (UniqueName: \"kubernetes.io/projected/13f96842-7202-4732-b60a-ff97323f64d3-kube-api-access-zbnjd\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.367931 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.367940    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47tw2\" (UniqueName: \"kubernetes.io/projected/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-kube-api-access-47tw2\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.367931 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.367949    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.368199 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.367957    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e5dcdb4-d4a5-44ca-a735-44521157a2ca-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.368199 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.367965    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/13f96842-7202-4732-b60a-ff97323f64d3-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:50.980892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.980810    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff"
Sep 10 14:48:50.980892 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.980805    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pqsff" event={"ID":"1b0894a5-3327-4d8e-83a8-948993ddfe30","Type":"ContainerDied","Data":"d529b2fa48166191280fda8070ef6356d7d481a3655d81121c69b52f80ad5ee6"}
Sep 10 14:48:50.981357 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.980936    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d529b2fa48166191280fda8070ef6356d7d481a3655d81121c69b52f80ad5ee6"
Sep 10 14:48:50.982866 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.982849    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn"
Sep 10 14:48:50.982984 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.982885    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cb7tcn" event={"ID":"4e5dcdb4-d4a5-44ca-a735-44521157a2ca","Type":"ContainerDied","Data":"ef67f064cb477eaab28ce3fbaa511d7dc6d3b70fea77df8eb370a09e84831a8f"}
Sep 10 14:48:50.982984 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.982925    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef67f064cb477eaab28ce3fbaa511d7dc6d3b70fea77df8eb370a09e84831a8f"
Sep 10 14:48:50.984890 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.984872    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl"
Sep 10 14:48:50.985001 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.984874    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847advkxl" event={"ID":"13f96842-7202-4732-b60a-ff97323f64d3","Type":"ContainerDied","Data":"5faca1e8dc1fd04dd10a2584e85785043f375e024691aaf655e596f4716e184e"}
Sep 10 14:48:50.985001 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.984984    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5faca1e8dc1fd04dd10a2584e85785043f375e024691aaf655e596f4716e184e"
Sep 10 14:48:50.987083 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.987050    2558 generic.go:356] "Generic (PLEG): container finished" podID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerID="cb885ead69f2ccceb76df0df70a7ce421faeee99f8f9c0ba5c136665cb091070" exitCode=0
Sep 10 14:48:50.987174 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:50.987119    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b" event={"ID":"6fba1b84-197e-43a8-b37b-b52f630d58f8","Type":"ContainerDied","Data":"cb885ead69f2ccceb76df0df70a7ce421faeee99f8f9c0ba5c136665cb091070"}
Sep 10 14:48:52.139002 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.138974    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:52.185118 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.185090    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxc2w\" (UniqueName: \"kubernetes.io/projected/6fba1b84-197e-43a8-b37b-b52f630d58f8-kube-api-access-fxc2w\") pod \"6fba1b84-197e-43a8-b37b-b52f630d58f8\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") "
Sep 10 14:48:52.185306 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.185159    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-bundle\") pod \"6fba1b84-197e-43a8-b37b-b52f630d58f8\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") "
Sep 10 14:48:52.185306 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.185195    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-util\") pod \"6fba1b84-197e-43a8-b37b-b52f630d58f8\" (UID: \"6fba1b84-197e-43a8-b37b-b52f630d58f8\") "
Sep 10 14:48:52.185680 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.185644    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-bundle" (OuterVolumeSpecName: "bundle") pod "6fba1b84-197e-43a8-b37b-b52f630d58f8" (UID: "6fba1b84-197e-43a8-b37b-b52f630d58f8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:52.187410 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.187386    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fba1b84-197e-43a8-b37b-b52f630d58f8-kube-api-access-fxc2w" (OuterVolumeSpecName: "kube-api-access-fxc2w") pod "6fba1b84-197e-43a8-b37b-b52f630d58f8" (UID: "6fba1b84-197e-43a8-b37b-b52f630d58f8"). InnerVolumeSpecName "kube-api-access-fxc2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:52.191262 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.191235    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-util" (OuterVolumeSpecName: "util") pod "6fba1b84-197e-43a8-b37b-b52f630d58f8" (UID: "6fba1b84-197e-43a8-b37b-b52f630d58f8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:52.286404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.286316    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:52.286404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.286346    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fba1b84-197e-43a8-b37b-b52f630d58f8-util\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:52.286404 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.286356    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxc2w\" (UniqueName: \"kubernetes.io/projected/6fba1b84-197e-43a8-b37b-b52f630d58f8-kube-api-access-fxc2w\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:48:52.999540 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.999504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b" event={"ID":"6fba1b84-197e-43a8-b37b-b52f630d58f8","Type":"ContainerDied","Data":"f1b9eb3532c5c30f4cbcadefa3a5dd2ba0bc92836d70849176bd2ce851348c27"}
Sep 10 14:48:52.999692 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.999542    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6d8j9b"
Sep 10 14:48:52.999692 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:48:52.999586    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1b9eb3532c5c30f4cbcadefa3a5dd2ba0bc92836d70849176bd2ce851348c27"
Sep 10 14:49:00.102277 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102240    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-556854d599-9hkpg"]
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102633    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f96842-7202-4732-b60a-ff97323f64d3" containerName="extract"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102648    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f96842-7202-4732-b60a-ff97323f64d3" containerName="extract"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102663    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerName="extract"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102668    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerName="extract"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102676    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f96842-7202-4732-b60a-ff97323f64d3" containerName="pull"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102681    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f96842-7202-4732-b60a-ff97323f64d3" containerName="pull"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102688    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerName="extract"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102693    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerName="extract"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102702    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f96842-7202-4732-b60a-ff97323f64d3" containerName="util"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102707    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f96842-7202-4732-b60a-ff97323f64d3" containerName="util"
Sep 10 14:49:00.102706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102713    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerName="extract"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102717    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerName="extract"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102723    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerName="pull"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102728    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerName="pull"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102734    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerName="pull"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102738    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerName="pull"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102744    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerName="util"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102748    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerName="util"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102758    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerName="util"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102763    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerName="util"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102769    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerName="util"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102773    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerName="util"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102779    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerName="pull"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102784    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerName="pull"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102841    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e5dcdb4-d4a5-44ca-a735-44521157a2ca" containerName="extract"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102848    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fba1b84-197e-43a8-b37b-b52f630d58f8" containerName="extract"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102853    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b0894a5-3327-4d8e-83a8-948993ddfe30" containerName="extract"
Sep 10 14:49:00.103085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.102861    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="13f96842-7202-4732-b60a-ff97323f64d3" containerName="extract"
Sep 10 14:49:00.105904 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.105887    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.109782 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:49:00.109757    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-556854d599-9hkpg\" is forbidden: User \"system:node:ip-10-0-142-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-142-253.ec2.internal' and this object" podUID="c2f5c934-5c72-4f90-a52a-ac0fae6c208d" pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.141029 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.140997    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556854d599-9hkpg"]
Sep 10 14:49:00.258929 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.258827    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-service-ca\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.259170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.259140    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-config\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.259336 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.259268    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7bmc\" (UniqueName: \"kubernetes.io/projected/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-kube-api-access-z7bmc\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.259336 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.259302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-serving-cert\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.259451 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.259356    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-trusted-ca-bundle\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.259496 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.259476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-oauth-config\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.259591 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.259575    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-oauth-serving-cert\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.360740 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.360640    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bmc\" (UniqueName: \"kubernetes.io/projected/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-kube-api-access-z7bmc\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.360740 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.360680    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-serving-cert\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.360740 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.360711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-trusted-ca-bundle\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.360740 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.360736    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-oauth-config\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.361076 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.360775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-oauth-serving-cert\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.361076 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.360825    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-service-ca\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.361076 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.360887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-config\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.361754 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.361697    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-oauth-serving-cert\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.361891 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.361813    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-config\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.361891 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.361820    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-trusted-ca-bundle\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.362022 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.361892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-service-ca\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.363664 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.363631    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-oauth-config\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.363807 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.363787    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-console-serving-cert\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.378142 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.378098    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bmc\" (UniqueName: \"kubernetes.io/projected/c2f5c934-5c72-4f90-a52a-ac0fae6c208d-kube-api-access-z7bmc\") pod \"console-556854d599-9hkpg\" (UID: \"c2f5c934-5c72-4f90-a52a-ac0fae6c208d\") " pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.414590 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.414554    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:00.716795 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:00.716770    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-556854d599-9hkpg"]
Sep 10 14:49:00.718686 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:49:00.718662    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f5c934_5c72_4f90_a52a_ac0fae6c208d.slice/crio-d46d23297d3d41bbc2665406de58825c878703d32e1d0b36da447f4352453364 WatchSource:0}: Error finding container d46d23297d3d41bbc2665406de58825c878703d32e1d0b36da447f4352453364: Status 404 returned error can't find the container with id d46d23297d3d41bbc2665406de58825c878703d32e1d0b36da447f4352453364
Sep 10 14:49:01.042179 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:01.042091    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556854d599-9hkpg" event={"ID":"c2f5c934-5c72-4f90-a52a-ac0fae6c208d","Type":"ContainerStarted","Data":"851f3d353dac6195d37d3ab07dce2c05ffdbbf274335c1cc3be577bdcceb726b"}
Sep 10 14:49:01.042179 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:01.042152    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-556854d599-9hkpg" event={"ID":"c2f5c934-5c72-4f90-a52a-ac0fae6c208d","Type":"ContainerStarted","Data":"d46d23297d3d41bbc2665406de58825c878703d32e1d0b36da447f4352453364"}
Sep 10 14:49:01.088071 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:01.088018    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-556854d599-9hkpg" podStartSLOduration=1.088002619 podStartE2EDuration="1.088002619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:49:01.082368241 +0000 UTC m=+385.569056144" watchObservedRunningTime="2026-09-10 14:49:01.088002619 +0000 UTC m=+385.574690579"
Sep 10 14:49:10.415706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:10.415658    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:10.416119 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:10.415733    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:10.420465 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:10.420441    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:11.101842 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.101806    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-556854d599-9hkpg"
Sep 10 14:49:11.201804 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.201770    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"]
Sep 10 14:49:11.205397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.205377    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.208274 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.208256    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 14:49:11.208542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.208524    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 14:49:11.208609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.208563    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 14:49:11.208609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.208528    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7tmbm\""
Sep 10 14:49:11.209131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.209113    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 14:49:11.218630 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.218607    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664f46c5dd-wxbmj"]
Sep 10 14:49:11.223993 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.223974    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"]
Sep 10 14:49:11.362593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.362500    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/59004aaa-96d5-4b27-a54f-003b649dfb36-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.362593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.362589    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/59004aaa-96d5-4b27-a54f-003b649dfb36-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.362796 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.362612    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6wtn\" (UniqueName: \"kubernetes.io/projected/59004aaa-96d5-4b27-a54f-003b649dfb36-kube-api-access-m6wtn\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.463803 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.463771    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/59004aaa-96d5-4b27-a54f-003b649dfb36-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.463803 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.463806    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6wtn\" (UniqueName: \"kubernetes.io/projected/59004aaa-96d5-4b27-a54f-003b649dfb36-kube-api-access-m6wtn\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.464236 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.463844    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/59004aaa-96d5-4b27-a54f-003b649dfb36-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.464429 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.464409    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/59004aaa-96d5-4b27-a54f-003b649dfb36-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.466322 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.466303    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/59004aaa-96d5-4b27-a54f-003b649dfb36-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.474614 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.474591    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6wtn\" (UniqueName: \"kubernetes.io/projected/59004aaa-96d5-4b27-a54f-003b649dfb36-kube-api-access-m6wtn\") pod \"kuadrant-console-plugin-fbdd4f445-bfzt6\" (UID: \"59004aaa-96d5-4b27-a54f-003b649dfb36\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.515623 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.515588    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"
Sep 10 14:49:11.793563 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:11.793535    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6"]
Sep 10 14:49:11.794568 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:49:11.794544    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59004aaa_96d5_4b27_a54f_003b649dfb36.slice/crio-cc1948bbc2b3f6e7e1d87e2faf2bae96ca0fc165db939ce1b2fb67e16c429a48 WatchSource:0}: Error finding container cc1948bbc2b3f6e7e1d87e2faf2bae96ca0fc165db939ce1b2fb67e16c429a48: Status 404 returned error can't find the container with id cc1948bbc2b3f6e7e1d87e2faf2bae96ca0fc165db939ce1b2fb67e16c429a48
Sep 10 14:49:12.104734 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:12.104649    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6" event={"ID":"59004aaa-96d5-4b27-a54f-003b649dfb36","Type":"ContainerStarted","Data":"cc1948bbc2b3f6e7e1d87e2faf2bae96ca0fc165db939ce1b2fb67e16c429a48"}
Sep 10 14:49:18.145923 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:18.145877    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6" event={"ID":"59004aaa-96d5-4b27-a54f-003b649dfb36","Type":"ContainerStarted","Data":"4cf6fdd6d9eb38b6a9434ac38da0339a5b33c8f860eb6cfdbdc5f0762e0067cc"}
Sep 10 14:49:18.173362 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:18.173308    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-bfzt6" podStartSLOduration=1.617199762 podStartE2EDuration="7.173292042s" totalImagesPullingTime="5.55609228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:49:18.172451582 +0000 UTC m=+402.659139475" watchObservedRunningTime="2026-09-10 14:49:18.173292042 +0000 UTC m=+402.659979933"
Sep 10 14:49:36.239783 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.239730    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-664f46c5dd-wxbmj" podUID="296457b2-459e-429c-9011-2a6a61cd05ea" containerName="console" containerID="cri-o://777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221" gracePeriod=15
Sep 10 14:49:36.490776 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.490718    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:49:36.592193 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592156    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-service-ca\") pod \"296457b2-459e-429c-9011-2a6a61cd05ea\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") "
Sep 10 14:49:36.592193 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592191    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-trusted-ca-bundle\") pod \"296457b2-459e-429c-9011-2a6a61cd05ea\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") "
Sep 10 14:49:36.592460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592269    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwx6l\" (UniqueName: \"kubernetes.io/projected/296457b2-459e-429c-9011-2a6a61cd05ea-kube-api-access-lwx6l\") pod \"296457b2-459e-429c-9011-2a6a61cd05ea\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") "
Sep 10 14:49:36.592460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592299    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-console-config\") pod \"296457b2-459e-429c-9011-2a6a61cd05ea\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") "
Sep 10 14:49:36.592460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592334    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-oauth-config\") pod \"296457b2-459e-429c-9011-2a6a61cd05ea\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") "
Sep 10 14:49:36.592460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592369    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-serving-cert\") pod \"296457b2-459e-429c-9011-2a6a61cd05ea\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") "
Sep 10 14:49:36.592648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592467    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-oauth-serving-cert\") pod \"296457b2-459e-429c-9011-2a6a61cd05ea\" (UID: \"296457b2-459e-429c-9011-2a6a61cd05ea\") "
Sep 10 14:49:36.592719 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592648    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-service-ca" (OuterVolumeSpecName: "service-ca") pod "296457b2-459e-429c-9011-2a6a61cd05ea" (UID: "296457b2-459e-429c-9011-2a6a61cd05ea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:49:36.592774 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592748    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-console-config" (OuterVolumeSpecName: "console-config") pod "296457b2-459e-429c-9011-2a6a61cd05ea" (UID: "296457b2-459e-429c-9011-2a6a61cd05ea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:49:36.592774 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592759    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "296457b2-459e-429c-9011-2a6a61cd05ea" (UID: "296457b2-459e-429c-9011-2a6a61cd05ea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:49:36.592880 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592773    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:49:36.592957 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.592939    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "296457b2-459e-429c-9011-2a6a61cd05ea" (UID: "296457b2-459e-429c-9011-2a6a61cd05ea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:49:36.594667 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.594633    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "296457b2-459e-429c-9011-2a6a61cd05ea" (UID: "296457b2-459e-429c-9011-2a6a61cd05ea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:49:36.594780 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.594671    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "296457b2-459e-429c-9011-2a6a61cd05ea" (UID: "296457b2-459e-429c-9011-2a6a61cd05ea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:49:36.594780 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.594695    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296457b2-459e-429c-9011-2a6a61cd05ea-kube-api-access-lwx6l" (OuterVolumeSpecName: "kube-api-access-lwx6l") pod "296457b2-459e-429c-9011-2a6a61cd05ea" (UID: "296457b2-459e-429c-9011-2a6a61cd05ea"). InnerVolumeSpecName "kube-api-access-lwx6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:49:36.693578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.693537    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwx6l\" (UniqueName: \"kubernetes.io/projected/296457b2-459e-429c-9011-2a6a61cd05ea-kube-api-access-lwx6l\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:49:36.693578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.693569    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-console-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:49:36.693578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.693579    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-oauth-config\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:49:36.693578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.693587    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/296457b2-459e-429c-9011-2a6a61cd05ea-console-serving-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:49:36.693836 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.693595    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-oauth-serving-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:49:36.693836 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:36.693603    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/296457b2-459e-429c-9011-2a6a61cd05ea-trusted-ca-bundle\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:49:37.243860 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.243823    2558 generic.go:356] "Generic (PLEG): container finished" podID="296457b2-459e-429c-9011-2a6a61cd05ea" containerID="777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221" exitCode=0
Sep 10 14:49:37.244272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.243872    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f46c5dd-wxbmj"
Sep 10 14:49:37.244272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.243910    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f46c5dd-wxbmj" event={"ID":"296457b2-459e-429c-9011-2a6a61cd05ea","Type":"ContainerDied","Data":"777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221"}
Sep 10 14:49:37.244272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.243961    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f46c5dd-wxbmj" event={"ID":"296457b2-459e-429c-9011-2a6a61cd05ea","Type":"ContainerDied","Data":"8df45e646acac63998744665bcccce1bea226693b3251199d3fde048d5a9559e"}
Sep 10 14:49:37.244272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.243987    2558 scope.go:117] "RemoveContainer" containerID="777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221"
Sep 10 14:49:37.253757 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.253739    2558 scope.go:117] "RemoveContainer" containerID="777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221"
Sep 10 14:49:37.254055 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:49:37.254037    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221\": container with ID starting with 777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221 not found: ID does not exist" containerID="777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221"
Sep 10 14:49:37.254125 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.254067    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221"} err="failed to get container status \"777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221\": rpc error: code = NotFound desc = could not find container \"777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221\": container with ID starting with 777939a822c873f348eba9b2ae2d6d20d1a949ff429ebb710ebc002e8a6e9221 not found: ID does not exist"
Sep 10 14:49:37.266371 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.266346    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664f46c5dd-wxbmj"]
Sep 10 14:49:37.271209 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:37.271186    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-664f46c5dd-wxbmj"]
Sep 10 14:49:38.018170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:38.018142    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="296457b2-459e-429c-9011-2a6a61cd05ea" path="/var/lib/kubelet/pods/296457b2-459e-429c-9011-2a6a61cd05ea/volumes"
Sep 10 14:49:58.381651 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.381575    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:49:58.382166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.381974    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296457b2-459e-429c-9011-2a6a61cd05ea" containerName="console"
Sep 10 14:49:58.382166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.381987    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="296457b2-459e-429c-9011-2a6a61cd05ea" containerName="console"
Sep 10 14:49:58.382166 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.382064    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="296457b2-459e-429c-9011-2a6a61cd05ea" containerName="console"
Sep 10 14:49:58.384817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.384801    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.387285 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.387263    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 14:49:58.394173 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.394148    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:49:58.423748 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.423716    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:49:58.478590 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.478552    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/680ffc7c-90dd-4fef-b8d4-1b978a41a79b-config-file\") pod \"limitador-limitador-84994f7ffc-5smxx\" (UID: \"680ffc7c-90dd-4fef-b8d4-1b978a41a79b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.478590 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.478592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmjnl\" (UniqueName: \"kubernetes.io/projected/680ffc7c-90dd-4fef-b8d4-1b978a41a79b-kube-api-access-xmjnl\") pod \"limitador-limitador-84994f7ffc-5smxx\" (UID: \"680ffc7c-90dd-4fef-b8d4-1b978a41a79b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.579628 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.579593    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/680ffc7c-90dd-4fef-b8d4-1b978a41a79b-config-file\") pod \"limitador-limitador-84994f7ffc-5smxx\" (UID: \"680ffc7c-90dd-4fef-b8d4-1b978a41a79b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.579792 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.579632    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmjnl\" (UniqueName: \"kubernetes.io/projected/680ffc7c-90dd-4fef-b8d4-1b978a41a79b-kube-api-access-xmjnl\") pod \"limitador-limitador-84994f7ffc-5smxx\" (UID: \"680ffc7c-90dd-4fef-b8d4-1b978a41a79b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.580243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.580201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/680ffc7c-90dd-4fef-b8d4-1b978a41a79b-config-file\") pod \"limitador-limitador-84994f7ffc-5smxx\" (UID: \"680ffc7c-90dd-4fef-b8d4-1b978a41a79b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.588180 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.588150    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmjnl\" (UniqueName: \"kubernetes.io/projected/680ffc7c-90dd-4fef-b8d4-1b978a41a79b-kube-api-access-xmjnl\") pod \"limitador-limitador-84994f7ffc-5smxx\" (UID: \"680ffc7c-90dd-4fef-b8d4-1b978a41a79b\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.695412 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.695383    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:49:58.790724 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.790693    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-zztmq"]
Sep 10 14:49:58.794384 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.794364    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq"
Sep 10 14:49:58.797113 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.797083    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w5vlc\""
Sep 10 14:49:58.802234 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.802202    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-zztmq"]
Sep 10 14:49:58.881673 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.881633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg82x\" (UniqueName: \"kubernetes.io/projected/26a3b5c7-0fca-46aa-987a-1b1dd2b2f703-kube-api-access-rg82x\") pod \"authorino-7bcfd8f9cd-zztmq\" (UID: \"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703\") " pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq"
Sep 10 14:49:58.974895 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.974867    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:49:58.977330 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:49:58.977304    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680ffc7c_90dd_4fef_b8d4_1b978a41a79b.slice/crio-ec0d081e7b2c950c672507d75fb712143f6f56b7c865e104358e6a6506ede236 WatchSource:0}: Error finding container ec0d081e7b2c950c672507d75fb712143f6f56b7c865e104358e6a6506ede236: Status 404 returned error can't find the container with id ec0d081e7b2c950c672507d75fb712143f6f56b7c865e104358e6a6506ede236
Sep 10 14:49:58.982631 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.982606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg82x\" (UniqueName: \"kubernetes.io/projected/26a3b5c7-0fca-46aa-987a-1b1dd2b2f703-kube-api-access-rg82x\") pod \"authorino-7bcfd8f9cd-zztmq\" (UID: \"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703\") " pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq"
Sep 10 14:49:58.994124 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:58.994094    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg82x\" (UniqueName: \"kubernetes.io/projected/26a3b5c7-0fca-46aa-987a-1b1dd2b2f703-kube-api-access-rg82x\") pod \"authorino-7bcfd8f9cd-zztmq\" (UID: \"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703\") " pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq"
Sep 10 14:49:59.105012 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:59.104977    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq"
Sep 10 14:49:59.375108 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:59.375072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx" event={"ID":"680ffc7c-90dd-4fef-b8d4-1b978a41a79b","Type":"ContainerStarted","Data":"ec0d081e7b2c950c672507d75fb712143f6f56b7c865e104358e6a6506ede236"}
Sep 10 14:49:59.375335 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:49:59.375320    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-zztmq"]
Sep 10 14:49:59.377382 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:49:59.377353    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26a3b5c7_0fca_46aa_987a_1b1dd2b2f703.slice/crio-97edc2fac73ca310952418ad517b530d34088f0d4373cc260409106abb15f1fb WatchSource:0}: Error finding container 97edc2fac73ca310952418ad517b530d34088f0d4373cc260409106abb15f1fb: Status 404 returned error can't find the container with id 97edc2fac73ca310952418ad517b530d34088f0d4373cc260409106abb15f1fb
Sep 10 14:50:00.382718 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:00.382673    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq" event={"ID":"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703","Type":"ContainerStarted","Data":"97edc2fac73ca310952418ad517b530d34088f0d4373cc260409106abb15f1fb"}
Sep 10 14:50:01.389554 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:01.389460    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx" event={"ID":"680ffc7c-90dd-4fef-b8d4-1b978a41a79b","Type":"ContainerStarted","Data":"da619de5cd2a681c14955d7de31bd69e8665d9a4b6ebc33dd5397f6075fc6a81"}
Sep 10 14:50:01.389999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:01.389593    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:50:01.408052 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:01.408007    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx" podStartSLOduration=1.304801888 podStartE2EDuration="3.40799312s" totalImagesPullingTime="2.103191232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:50:01.406942577 +0000 UTC m=+445.893630461" watchObservedRunningTime="2026-09-10 14:50:01.40799312 +0000 UTC m=+445.894681010"
Sep 10 14:50:02.168788 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:02.168746    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-zztmq"]
Sep 10 14:50:04.408711 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:04.408620    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq" event={"ID":"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703","Type":"ContainerStarted","Data":"94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8"}
Sep 10 14:50:04.408711 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:04.408687    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq" podUID="26a3b5c7-0fca-46aa-987a-1b1dd2b2f703" containerName="authorino" containerID="cri-o://94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8" gracePeriod=30
Sep 10 14:50:04.425231 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:04.425172    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq" podStartSLOduration=1.8142681619999999 podStartE2EDuration="6.425156555s" totalImagesPullingTime="4.610888393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:50:04.422925415 +0000 UTC m=+448.909613298" watchObservedRunningTime="2026-09-10 14:50:04.425156555 +0000 UTC m=+448.911844446"
Sep 10 14:50:04.647962 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:04.647936    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq"
Sep 10 14:50:04.733474 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:04.733438    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg82x\" (UniqueName: \"kubernetes.io/projected/26a3b5c7-0fca-46aa-987a-1b1dd2b2f703-kube-api-access-rg82x\") pod \"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703\" (UID: \"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703\") "
Sep 10 14:50:04.735819 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:04.735792    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a3b5c7-0fca-46aa-987a-1b1dd2b2f703-kube-api-access-rg82x" (OuterVolumeSpecName: "kube-api-access-rg82x") pod "26a3b5c7-0fca-46aa-987a-1b1dd2b2f703" (UID: "26a3b5c7-0fca-46aa-987a-1b1dd2b2f703"). InnerVolumeSpecName "kube-api-access-rg82x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:04.834090 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:04.834057    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg82x\" (UniqueName: \"kubernetes.io/projected/26a3b5c7-0fca-46aa-987a-1b1dd2b2f703-kube-api-access-rg82x\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:05.416533 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.416494    2558 generic.go:356] "Generic (PLEG): container finished" podID="26a3b5c7-0fca-46aa-987a-1b1dd2b2f703" containerID="94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8" exitCode=0
Sep 10 14:50:05.416921 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.416541    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq"
Sep 10 14:50:05.416921 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.416580    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq" event={"ID":"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703","Type":"ContainerDied","Data":"94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8"}
Sep 10 14:50:05.416921 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.416623    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-zztmq" event={"ID":"26a3b5c7-0fca-46aa-987a-1b1dd2b2f703","Type":"ContainerDied","Data":"97edc2fac73ca310952418ad517b530d34088f0d4373cc260409106abb15f1fb"}
Sep 10 14:50:05.416921 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.416642    2558 scope.go:117] "RemoveContainer" containerID="94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8"
Sep 10 14:50:05.425506 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.425489    2558 scope.go:117] "RemoveContainer" containerID="94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8"
Sep 10 14:50:05.425764 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:50:05.425746    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8\": container with ID starting with 94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8 not found: ID does not exist" containerID="94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8"
Sep 10 14:50:05.425810 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.425772    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8"} err="failed to get container status \"94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8\": rpc error: code = NotFound desc = could not find container \"94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8\": container with ID starting with 94e313b866a0ed5fd9591f0548ef9ee68e904b2dbe16fb7712b4daaf94df75f8 not found: ID does not exist"
Sep 10 14:50:05.438098 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.438074    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-zztmq"]
Sep 10 14:50:05.441716 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:05.441697    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-zztmq"]
Sep 10 14:50:06.020043 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:06.020013    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a3b5c7-0fca-46aa-987a-1b1dd2b2f703" path="/var/lib/kubelet/pods/26a3b5c7-0fca-46aa-987a-1b1dd2b2f703/volumes"
Sep 10 14:50:12.396623 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:12.396594    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-5smxx"
Sep 10 14:50:26.801342 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.801304    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-52nwd"]
Sep 10 14:50:26.801774 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.801724    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a3b5c7-0fca-46aa-987a-1b1dd2b2f703" containerName="authorino"
Sep 10 14:50:26.801774 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.801739    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a3b5c7-0fca-46aa-987a-1b1dd2b2f703" containerName="authorino"
Sep 10 14:50:26.801846 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.801809    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a3b5c7-0fca-46aa-987a-1b1dd2b2f703" containerName="authorino"
Sep 10 14:50:26.806188 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.806169    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-52nwd"
Sep 10 14:50:26.809027 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.809001    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w5vlc\""
Sep 10 14:50:26.811567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.811544    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-52nwd"]
Sep 10 14:50:26.863366 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.863328    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-52nwd"]
Sep 10 14:50:26.863633 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:50:26.863610    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-p8r2r], unattached volumes=[], failed to process volumes=[kube-api-access-p8r2r]: context canceled" pod="kuadrant-system/authorino-557f876f69-52nwd" podUID="bcf2e9c3-10dc-4a06-ad06-3a7f9125f406"
Sep 10 14:50:26.894251 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.894206    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-v4txs"]
Sep 10 14:50:26.897465 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.897445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:26.899788 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.899767    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 14:50:26.904753 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.904731    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-v4txs"]
Sep 10 14:50:26.934689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:26.934654    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8r2r\" (UniqueName: \"kubernetes.io/projected/bcf2e9c3-10dc-4a06-ad06-3a7f9125f406-kube-api-access-p8r2r\") pod \"authorino-557f876f69-52nwd\" (UID: \"bcf2e9c3-10dc-4a06-ad06-3a7f9125f406\") " pod="kuadrant-system/authorino-557f876f69-52nwd"
Sep 10 14:50:27.035846 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.035809    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-tls-cert\") pod \"authorino-6d4dfc649f-v4txs\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") " pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.036041 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.035896    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8r2r\" (UniqueName: \"kubernetes.io/projected/bcf2e9c3-10dc-4a06-ad06-3a7f9125f406-kube-api-access-p8r2r\") pod \"authorino-557f876f69-52nwd\" (UID: \"bcf2e9c3-10dc-4a06-ad06-3a7f9125f406\") " pod="kuadrant-system/authorino-557f876f69-52nwd"
Sep 10 14:50:27.036041 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.035995    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7twxm\" (UniqueName: \"kubernetes.io/projected/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-kube-api-access-7twxm\") pod \"authorino-6d4dfc649f-v4txs\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") " pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.038083 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.038062    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-v4txs"]
Sep 10 14:50:27.038286 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:50:27.038269    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7twxm tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-v4txs" podUID="60ecac74-bde2-47ac-a5c4-6c74c8aaa27d"
Sep 10 14:50:27.049668 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.049647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8r2r\" (UniqueName: \"kubernetes.io/projected/bcf2e9c3-10dc-4a06-ad06-3a7f9125f406-kube-api-access-p8r2r\") pod \"authorino-557f876f69-52nwd\" (UID: \"bcf2e9c3-10dc-4a06-ad06-3a7f9125f406\") " pod="kuadrant-system/authorino-557f876f69-52nwd"
Sep 10 14:50:27.136931 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.136834    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7twxm\" (UniqueName: \"kubernetes.io/projected/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-kube-api-access-7twxm\") pod \"authorino-6d4dfc649f-v4txs\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") " pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.136931 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.136891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-tls-cert\") pod \"authorino-6d4dfc649f-v4txs\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") " pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.139647 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.139619    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-tls-cert\") pod \"authorino-6d4dfc649f-v4txs\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") " pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.145086 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.145061    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7twxm\" (UniqueName: \"kubernetes.io/projected/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-kube-api-access-7twxm\") pod \"authorino-6d4dfc649f-v4txs\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") " pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.537434 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.537403    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.537721 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.537707    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-52nwd"
Sep 10 14:50:27.549925 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.549895    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:27.552987 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.552959    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-52nwd"
Sep 10 14:50:27.642645 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.642615    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7twxm\" (UniqueName: \"kubernetes.io/projected/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-kube-api-access-7twxm\") pod \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") "
Sep 10 14:50:27.642645 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.642650    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-tls-cert\") pod \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\" (UID: \"60ecac74-bde2-47ac-a5c4-6c74c8aaa27d\") "
Sep 10 14:50:27.642886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.642693    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8r2r\" (UniqueName: \"kubernetes.io/projected/bcf2e9c3-10dc-4a06-ad06-3a7f9125f406-kube-api-access-p8r2r\") pod \"bcf2e9c3-10dc-4a06-ad06-3a7f9125f406\" (UID: \"bcf2e9c3-10dc-4a06-ad06-3a7f9125f406\") "
Sep 10 14:50:27.644939 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.644911    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "60ecac74-bde2-47ac-a5c4-6c74c8aaa27d" (UID: "60ecac74-bde2-47ac-a5c4-6c74c8aaa27d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:50:27.645048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.644908    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-kube-api-access-7twxm" (OuterVolumeSpecName: "kube-api-access-7twxm") pod "60ecac74-bde2-47ac-a5c4-6c74c8aaa27d" (UID: "60ecac74-bde2-47ac-a5c4-6c74c8aaa27d"). InnerVolumeSpecName "kube-api-access-7twxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:27.645048 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.644972    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf2e9c3-10dc-4a06-ad06-3a7f9125f406-kube-api-access-p8r2r" (OuterVolumeSpecName: "kube-api-access-p8r2r") pod "bcf2e9c3-10dc-4a06-ad06-3a7f9125f406" (UID: "bcf2e9c3-10dc-4a06-ad06-3a7f9125f406"). InnerVolumeSpecName "kube-api-access-p8r2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:27.743338 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.743306    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7twxm\" (UniqueName: \"kubernetes.io/projected/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-kube-api-access-7twxm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:27.743338 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.743332    2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d-tls-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:27.743338 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:27.743344    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8r2r\" (UniqueName: \"kubernetes.io/projected/bcf2e9c3-10dc-4a06-ad06-3a7f9125f406-kube-api-access-p8r2r\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:50:28.542135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:28.542104    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-52nwd"
Sep 10 14:50:28.542610 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:28.542111    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-v4txs"
Sep 10 14:50:28.571567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:28.571538    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-52nwd"]
Sep 10 14:50:28.575272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:28.575247    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-52nwd"]
Sep 10 14:50:28.596116 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:28.596088    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-v4txs"]
Sep 10 14:50:28.603806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:28.603780    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-v4txs"]
Sep 10 14:50:30.018818 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:30.018782    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60ecac74-bde2-47ac-a5c4-6c74c8aaa27d" path="/var/lib/kubelet/pods/60ecac74-bde2-47ac-a5c4-6c74c8aaa27d/volumes"
Sep 10 14:50:30.019397 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:50:30.019120    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf2e9c3-10dc-4a06-ad06-3a7f9125f406" path="/var/lib/kubelet/pods/bcf2e9c3-10dc-4a06-ad06-3a7f9125f406/volumes"
Sep 10 14:51:13.542572 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:13.542536    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:51:30.076046 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.075963    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"]
Sep 10 14:51:30.079463 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.079447    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.083295 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.083259    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5tt4z\""
Sep 10 14:51:30.083295 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.083287    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 14:51:30.083553 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.083301    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 14:51:30.083553 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.083288    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 14:51:30.090517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.090495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"]
Sep 10 14:51:30.188623 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.188580    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.188623 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.188625    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.188837 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.188650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.188837 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.188697    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.188837 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.188801    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68nc\" (UniqueName: \"kubernetes.io/projected/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-kube-api-access-f68nc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.188837 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.188825    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.289648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.289606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.289648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.289645    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.289903 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.289661    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.289903 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.289675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.289903 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.289738    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68nc\" (UniqueName: \"kubernetes.io/projected/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-kube-api-access-f68nc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.289903 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.289771    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.290116 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.290059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.290164 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.290130    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.290164 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.290157    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.292140 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.292108    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.292448 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.292431    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.297258 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.297237    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68nc\" (UniqueName: \"kubernetes.io/projected/7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64-kube-api-access-f68nc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc\" (UID: \"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.390780 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.390680    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:30.566799 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.566756    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:51:30.669466 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.669445    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"]
Sep 10 14:51:30.671450 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:51:30.671423    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8a3ce1_9ec9_435b_83e7_442aa0ce8f64.slice/crio-aff6b5e8115da1b7aaee012204494a5566e4e83a66fd854c0ceac852490a842e WatchSource:0}: Error finding container aff6b5e8115da1b7aaee012204494a5566e4e83a66fd854c0ceac852490a842e: Status 404 returned error can't find the container with id aff6b5e8115da1b7aaee012204494a5566e4e83a66fd854c0ceac852490a842e
Sep 10 14:51:30.879775 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:30.879742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc" event={"ID":"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64","Type":"ContainerStarted","Data":"aff6b5e8115da1b7aaee012204494a5566e4e83a66fd854c0ceac852490a842e"}
Sep 10 14:51:31.887039 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:31.887000    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc" event={"ID":"7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64","Type":"ContainerStarted","Data":"b4b2b3fde3cf7b602f3dc0d1b67abd61fcd74b5ebc1d896af2715904b998e3cb"}
Sep 10 14:51:31.906407 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:31.906356    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc" podStartSLOduration=0.793990322 podStartE2EDuration="1.906331576s" totalImagesPullingTime="1.112341254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:51:31.904446165 +0000 UTC m=+536.391134057" watchObservedRunningTime="2026-09-10 14:51:31.906331576 +0000 UTC m=+536.393019464"
Sep 10 14:51:40.391705 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:40.391657    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:40.392157 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:40.391737    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:40.404131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:40.404098    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:40.946605 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:40.946572    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc"
Sep 10 14:51:43.066669 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:51:43.066627    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:52:07.073101 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.073056    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"]
Sep 10 14:52:07.078088 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.078068    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.080648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.080617    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 14:52:07.085379 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.085357    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"]
Sep 10 14:52:07.220667 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.220634    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.220667 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.220666    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.220918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.220687    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.220918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.220718    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.220918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.220822    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4117a54b-cb10-49e9-9ad2-9056d26f93d0-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.220918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.220862    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hz77\" (UniqueName: \"kubernetes.io/projected/4117a54b-cb10-49e9-9ad2-9056d26f93d0-kube-api-access-6hz77\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.321378 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321330    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.321378 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.321609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.321609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321425    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.321609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321477    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4117a54b-cb10-49e9-9ad2-9056d26f93d0-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.321609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hz77\" (UniqueName: \"kubernetes.io/projected/4117a54b-cb10-49e9-9ad2-9056d26f93d0-kube-api-access-6hz77\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.321851 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321808    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.322096 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.322096 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.321867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.324018 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.323964    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4117a54b-cb10-49e9-9ad2-9056d26f93d0-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.324111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.324065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4117a54b-cb10-49e9-9ad2-9056d26f93d0-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.331043 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.331020    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hz77\" (UniqueName: \"kubernetes.io/projected/4117a54b-cb10-49e9-9ad2-9056d26f93d0-kube-api-access-6hz77\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c\" (UID: \"4117a54b-cb10-49e9-9ad2-9056d26f93d0\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.390080 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.390044    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:07.663599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.663577    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"]
Sep 10 14:52:07.665591 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:52:07.665561    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4117a54b_cb10_49e9_9ad2_9056d26f93d0.slice/crio-2ef325806cb223e6c68e3cc7af3be7bf9c3ddcf2c119590f8a85c968a8884663 WatchSource:0}: Error finding container 2ef325806cb223e6c68e3cc7af3be7bf9c3ddcf2c119590f8a85c968a8884663: Status 404 returned error can't find the container with id 2ef325806cb223e6c68e3cc7af3be7bf9c3ddcf2c119590f8a85c968a8884663
Sep 10 14:52:07.969729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:07.969697    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:52:08.082670 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:08.082634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c" event={"ID":"4117a54b-cb10-49e9-9ad2-9056d26f93d0","Type":"ContainerStarted","Data":"f621b5a438e8265f8df7c498922a50370d928bf3ec116cf4014dafd85ea90e3e"}
Sep 10 14:52:08.083056 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:08.082685    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c" event={"ID":"4117a54b-cb10-49e9-9ad2-9056d26f93d0","Type":"ContainerStarted","Data":"2ef325806cb223e6c68e3cc7af3be7bf9c3ddcf2c119590f8a85c968a8884663"}
Sep 10 14:52:08.101085 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:08.101027    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c" podStartSLOduration=0.876888951 podStartE2EDuration="1.101011782s" totalImagesPullingTime="224.122831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:52:08.098213008 +0000 UTC m=+572.584900900" watchObservedRunningTime="2026-09-10 14:52:08.101011782 +0000 UTC m=+572.587699710"
Sep 10 14:52:17.390987 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:17.390952    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:17.391420 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:17.391037    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:17.403545 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:17.403521    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:18.148178 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:18.148140    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c"
Sep 10 14:52:24.467710 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:24.467669    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:52:39.666037 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:52:39.666003    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:53:03.078868 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:03.078789    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:53:08.467113 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.467077    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"]
Sep 10 14:53:08.470511 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.470491    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.473096 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.473072    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 14:53:08.479823 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.479788    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"]
Sep 10 14:53:08.566858 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.566823    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.566858 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.566866    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.567119 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.566951    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.567119 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.567031    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.567119 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.567052    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5531299-f326-4594-a2ac-be4d52a0c789-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.567119 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.567113    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8x4x\" (UniqueName: \"kubernetes.io/projected/f5531299-f326-4594-a2ac-be4d52a0c789-kube-api-access-x8x4x\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668334 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668294    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668344    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668400    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668432    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668457    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5531299-f326-4594-a2ac-be4d52a0c789-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668531 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668515    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8x4x\" (UniqueName: \"kubernetes.io/projected/f5531299-f326-4594-a2ac-be4d52a0c789-kube-api-access-x8x4x\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668785    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668801    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.668907 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.668848    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.670903 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.670873    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5531299-f326-4594-a2ac-be4d52a0c789-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.671153 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.671135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5531299-f326-4594-a2ac-be4d52a0c789-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.676714 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.676696    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8x4x\" (UniqueName: \"kubernetes.io/projected/f5531299-f326-4594-a2ac-be4d52a0c789-kube-api-access-x8x4x\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv\" (UID: \"f5531299-f326-4594-a2ac-be4d52a0c789\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:08.780804 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:08.780702    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:09.056651 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:09.056625    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"]
Sep 10 14:53:09.058095 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:53:09.058069    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5531299_f326_4594_a2ac_be4d52a0c789.slice/crio-7c61a907a5606a142d10dd27802341fe2a8db269f4e4c5abf137512b4c44c2a3 WatchSource:0}: Error finding container 7c61a907a5606a142d10dd27802341fe2a8db269f4e4c5abf137512b4c44c2a3: Status 404 returned error can't find the container with id 7c61a907a5606a142d10dd27802341fe2a8db269f4e4c5abf137512b4c44c2a3
Sep 10 14:53:09.410175 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:09.410139    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv" event={"ID":"f5531299-f326-4594-a2ac-be4d52a0c789","Type":"ContainerStarted","Data":"74b6b71cbab387c6ad24e2db3a68ab1a499aef48181ce318360b426217b75a19"}
Sep 10 14:53:09.410398 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:09.410189    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv" event={"ID":"f5531299-f326-4594-a2ac-be4d52a0c789","Type":"ContainerStarted","Data":"7c61a907a5606a142d10dd27802341fe2a8db269f4e4c5abf137512b4c44c2a3"}
Sep 10 14:53:09.428809 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:09.428755    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv" podStartSLOduration=1.177453365 podStartE2EDuration="1.42873737s" totalImagesPullingTime="251.284005ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:53:09.426868702 +0000 UTC m=+633.913556598" watchObservedRunningTime="2026-09-10 14:53:09.42873737 +0000 UTC m=+633.915425263"
Sep 10 14:53:10.967942 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:10.967906    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:53:18.781799 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:18.781748    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:18.781799 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:18.781808    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:18.794320 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:18.794295    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:53:19.474180 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:53:19.474152    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv"
Sep 10 14:54:40.946315 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:40.946239    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-795b565bc7-s2f8c"]
Sep 10 14:54:40.949597 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:40.949581    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:40.952127 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:40.952105    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 14:54:40.953023 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:40.953008    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w5vlc\""
Sep 10 14:54:40.957190 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:40.957163    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-795b565bc7-s2f8c"]
Sep 10 14:54:41.016314 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.016282    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfbhd\" (UniqueName: \"kubernetes.io/projected/0bbdb5d7-2872-4e0f-89d0-22589b15621f-kube-api-access-sfbhd\") pod \"authorino-795b565bc7-s2f8c\" (UID: \"0bbdb5d7-2872-4e0f-89d0-22589b15621f\") " pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:41.016469 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.016421    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bbdb5d7-2872-4e0f-89d0-22589b15621f-tls-cert\") pod \"authorino-795b565bc7-s2f8c\" (UID: \"0bbdb5d7-2872-4e0f-89d0-22589b15621f\") " pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:41.117058 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.117020    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bbdb5d7-2872-4e0f-89d0-22589b15621f-tls-cert\") pod \"authorino-795b565bc7-s2f8c\" (UID: \"0bbdb5d7-2872-4e0f-89d0-22589b15621f\") " pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:41.117265 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.117062    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfbhd\" (UniqueName: \"kubernetes.io/projected/0bbdb5d7-2872-4e0f-89d0-22589b15621f-kube-api-access-sfbhd\") pod \"authorino-795b565bc7-s2f8c\" (UID: \"0bbdb5d7-2872-4e0f-89d0-22589b15621f\") " pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:41.119640 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.119619    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0bbdb5d7-2872-4e0f-89d0-22589b15621f-tls-cert\") pod \"authorino-795b565bc7-s2f8c\" (UID: \"0bbdb5d7-2872-4e0f-89d0-22589b15621f\") " pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:41.125795 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.125768    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfbhd\" (UniqueName: \"kubernetes.io/projected/0bbdb5d7-2872-4e0f-89d0-22589b15621f-kube-api-access-sfbhd\") pod \"authorino-795b565bc7-s2f8c\" (UID: \"0bbdb5d7-2872-4e0f-89d0-22589b15621f\") " pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:41.259793 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.259697    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-795b565bc7-s2f8c"
Sep 10 14:54:41.528697 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.528660    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-795b565bc7-s2f8c"]
Sep 10 14:54:41.530180 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:54:41.530152    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bbdb5d7_2872_4e0f_89d0_22589b15621f.slice/crio-86e362a1ecf35d1a1de5cd52ab1914dfb00f32d501a914b2c93e320eb96bc309 WatchSource:0}: Error finding container 86e362a1ecf35d1a1de5cd52ab1914dfb00f32d501a914b2c93e320eb96bc309: Status 404 returned error can't find the container with id 86e362a1ecf35d1a1de5cd52ab1914dfb00f32d501a914b2c93e320eb96bc309
Sep 10 14:54:41.531584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.531563    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:54:41.899982 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:41.899899    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-795b565bc7-s2f8c" event={"ID":"0bbdb5d7-2872-4e0f-89d0-22589b15621f","Type":"ContainerStarted","Data":"86e362a1ecf35d1a1de5cd52ab1914dfb00f32d501a914b2c93e320eb96bc309"}
Sep 10 14:54:42.909527 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:42.909494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-795b565bc7-s2f8c" event={"ID":"0bbdb5d7-2872-4e0f-89d0-22589b15621f","Type":"ContainerStarted","Data":"1f229e7ded6b45250ba3461709987ab5170d9c0ebdb76ac1ba5cb09fb1f13f56"}
Sep 10 14:54:42.927203 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:54:42.927152    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-795b565bc7-s2f8c" podStartSLOduration=2.021477537 podStartE2EDuration="2.927134047s" totalImagesPullingTime="905.65651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:54:42.924300833 +0000 UTC m=+727.410988725" watchObservedRunningTime="2026-09-10 14:54:42.927134047 +0000 UTC m=+727.413821937"
Sep 10 14:55:04.871886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:04.871851    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:55:20.762504 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.762475    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"]
Sep 10 14:55:20.766398 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.766380    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.769419 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.769396    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:55:20.775124 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.775105    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f7fc2b9f-openshift-default-dockercfg-mvfjg\""
Sep 10 14:55:20.786516 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.786495    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"]
Sep 10 14:55:20.887982 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.887947    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-certs\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888190 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888000    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-podinfo\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888190 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888020    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nccx\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-kube-api-access-4nccx\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888190 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888102    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istiod-ca-cert\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888190 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888144    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-socket\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888469 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888256    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-credential-socket\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888469 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-token\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888469 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888308    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-envoy\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.888469 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.888360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-data\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989675 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-podinfo\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989675 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nccx\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-kube-api-access-4nccx\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989711    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istiod-ca-cert\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989740    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-socket\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989789    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-credential-socket\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989811    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-token\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989847    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-envoy\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.989953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989901    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-data\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.990258 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.989963    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-certs\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.990369 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.990346    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-credential-socket\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.990439 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.990412    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-socket\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.990497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.990468    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-certs\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.990638 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.990620    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istiod-ca-cert\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.990816 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.990795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-data\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.992327 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.992300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-envoy\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:20.992438 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:20.992420    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-podinfo\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:21.000187 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:21.000166    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-token\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:21.000310 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:21.000167    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nccx\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-kube-api-access-4nccx\") pod \"e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") " pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:21.076855 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:21.076767    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:21.363650 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:21.363624    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"]
Sep 10 14:55:21.365857 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:55:21.365823    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f86912e_43ed_40fe_921e_8a0da7e2d7e7.slice/crio-298f60b058d450b972b4cf4915691a63e21754ef4cf707fa11ae91bfdca2d0f0 WatchSource:0}: Error finding container 298f60b058d450b972b4cf4915691a63e21754ef4cf707fa11ae91bfdca2d0f0: Status 404 returned error can't find the container with id 298f60b058d450b972b4cf4915691a63e21754ef4cf707fa11ae91bfdca2d0f0
Sep 10 14:55:22.124291 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:22.124248    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm" event={"ID":"0f86912e-43ed-40fe-921e-8a0da7e2d7e7","Type":"ContainerStarted","Data":"298f60b058d450b972b4cf4915691a63e21754ef4cf707fa11ae91bfdca2d0f0"}
Sep 10 14:55:23.829911 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:23.829864    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:55:23.830209 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:23.829948    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:55:23.830209 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:23.829985    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:55:24.138201 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:24.138095    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm" event={"ID":"0f86912e-43ed-40fe-921e-8a0da7e2d7e7","Type":"ContainerStarted","Data":"0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb"}
Sep 10 14:55:24.161059 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:24.161005    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm" podStartSLOduration=1.699368882 podStartE2EDuration="4.160987104s" totalImagesPullingTime="2.461618222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:24.15952256 +0000 UTC m=+768.646210480" watchObservedRunningTime="2026-09-10 14:55:24.160987104 +0000 UTC m=+768.647674994"
Sep 10 14:55:24.681213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:24.681179    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"]
Sep 10 14:55:25.077859 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:25.077819    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:25.082887 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:25.082862    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:25.143137 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:25.143111    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:25.143328 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:25.143112    2558 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm" secret="" err="secret \"e2e-ait-f7fc2b9f-openshift-default-dockercfg-mvfjg\" not found"
Sep 10 14:55:25.144056 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:25.144037    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:26.148756 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:26.148707    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm" podUID="0f86912e-43ed-40fe-921e-8a0da7e2d7e7" containerName="istio-proxy" containerID="cri-o://0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb" gracePeriod=30
Sep 10 14:55:28.668931 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.668899    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"]
Sep 10 14:55:28.672760 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.672736    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.675419 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.675399    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-b56e8eb3-kserve-self-signed-certs\""
Sep 10 14:55:28.682497 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.682473    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"]
Sep 10 14:55:28.757601 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.757571    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-home\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.757601 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.757606    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-model-cache\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.757811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.757694    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tls-certs\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.757811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.757728    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9tvz\" (UniqueName: \"kubernetes.io/projected/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-kube-api-access-m9tvz\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.757811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.757753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-dshm\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.757811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.757796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tmp-dir\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.858696 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.858660    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tls-certs\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.858696 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.858697    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9tvz\" (UniqueName: \"kubernetes.io/projected/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-kube-api-access-m9tvz\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.858901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.858724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-dshm\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.858901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.858786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tmp-dir\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.858901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.858848    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-home\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.858901 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.858875    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-model-cache\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.859276 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.859249    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-model-cache\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.859276 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.859258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tmp-dir\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.859451 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.859333    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-home\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.861261 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.861209    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-dshm\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.861371 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.861356    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tls-certs\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.868716 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.868691    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9tvz\" (UniqueName: \"kubernetes.io/projected/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-kube-api-access-m9tvz\") pod \"e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") " pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:28.982600 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:28.982562    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:29.257610 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:29.257581    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"]
Sep 10 14:55:29.259040 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:55:29.259016    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72fdc00e_1d59_45bb_9010_c83cb0d1fb0b.slice/crio-e4b7be56d0fe5c59204bf9fedec6c32cd26c6a85ed205e139867cf858fc9535a WatchSource:0}: Error finding container e4b7be56d0fe5c59204bf9fedec6c32cd26c6a85ed205e139867cf858fc9535a: Status 404 returned error can't find the container with id e4b7be56d0fe5c59204bf9fedec6c32cd26c6a85ed205e139867cf858fc9535a
Sep 10 14:55:30.173243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:30.173182    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8" event={"ID":"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b","Type":"ContainerStarted","Data":"50cc7efe16ad56cdbe128a3267ed2e5f8463bb64d7821bcb746b5e4d64becfc4"}
Sep 10 14:55:30.173243 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:30.173234    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8" event={"ID":"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b","Type":"ContainerStarted","Data":"e4b7be56d0fe5c59204bf9fedec6c32cd26c6a85ed205e139867cf858fc9535a"}
Sep 10 14:55:30.196422 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:30.196371    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8" podStartSLOduration=2.196355575 podStartE2EDuration="2.196355575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:30.192789179 +0000 UTC m=+774.679477071" watchObservedRunningTime="2026-09-10 14:55:30.196355575 +0000 UTC m=+774.683043470"
Sep 10 14:55:31.399703 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.399677    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:31.484556 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484518    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-token\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.484556 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484554    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-data\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.484769 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484580    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-envoy\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.484769 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484617    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-certs\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.484769 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484738    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-credential-socket\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.484916 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484791    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nccx\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-kube-api-access-4nccx\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.484916 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484868    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-socket\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.484916 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484848    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:55:31.484916 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484900    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-podinfo\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.485100 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484927    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:55:31.485100 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.484964    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:55:31.485100 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.485023    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istiod-ca-cert\") pod \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\" (UID: \"0f86912e-43ed-40fe-921e-8a0da7e2d7e7\") "
Sep 10 14:55:31.485307 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.485157    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:55:31.485498 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.485443    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-data\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.485619 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.485446    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:55:31.485749 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.485592    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.485749 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.485640    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-credential-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.485749 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.485655    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-workload-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.487251 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.487207    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-kube-api-access-4nccx" (OuterVolumeSpecName: "kube-api-access-4nccx") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "kube-api-access-4nccx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:55:31.487251 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.487237    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:55:31.487480 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.487461    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:55:31.487480 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.487466    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f86912e-43ed-40fe-921e-8a0da7e2d7e7" (UID: "0f86912e-43ed-40fe-921e-8a0da7e2d7e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:55:31.588447 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.586922    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istiod-ca-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.588447 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.586959    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-token\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.588447 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.586974    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-envoy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.588447 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.586995    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nccx\" (UniqueName: \"kubernetes.io/projected/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-kube-api-access-4nccx\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.588447 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.587012    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f86912e-43ed-40fe-921e-8a0da7e2d7e7-istio-podinfo\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:55:31.702041 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:31.702005    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:55:32.185796 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.185756    2558 generic.go:356] "Generic (PLEG): container finished" podID="0f86912e-43ed-40fe-921e-8a0da7e2d7e7" containerID="0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb" exitCode=0
Sep 10 14:55:32.185796 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.185799    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm" event={"ID":"0f86912e-43ed-40fe-921e-8a0da7e2d7e7","Type":"ContainerDied","Data":"0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb"}
Sep 10 14:55:32.186073 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.185830    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"
Sep 10 14:55:32.186073 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.185861    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm" event={"ID":"0f86912e-43ed-40fe-921e-8a0da7e2d7e7","Type":"ContainerDied","Data":"298f60b058d450b972b4cf4915691a63e21754ef4cf707fa11ae91bfdca2d0f0"}
Sep 10 14:55:32.186073 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.185891    2558 scope.go:117] "RemoveContainer" containerID="0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb"
Sep 10 14:55:32.195868 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.195849    2558 scope.go:117] "RemoveContainer" containerID="0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb"
Sep 10 14:55:32.196158 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:55:32.196141    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb\": container with ID starting with 0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb not found: ID does not exist" containerID="0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb"
Sep 10 14:55:32.196205 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.196168    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb"} err="failed to get container status \"0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb\": rpc error: code = NotFound desc = could not find container \"0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb\": container with ID starting with 0327cdc38b04f97e28e26d5d801d38fe522d38706d991eba963c32075fa98ccb not found: ID does not exist"
Sep 10 14:55:32.214904 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.214862    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"]
Sep 10 14:55:32.217088 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:32.217061    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f7fc2b9f-openshift-default-99c565b6-gkgpm"]
Sep 10 14:55:34.019204 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:34.019175    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f86912e-43ed-40fe-921e-8a0da7e2d7e7" path="/var/lib/kubelet/pods/0f86912e-43ed-40fe-921e-8a0da7e2d7e7/volumes"
Sep 10 14:55:38.982882 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:38.982843    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:38.983341 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:38.982933    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:38.995097 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:38.995075    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:39.236647 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:39.236561    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:55:41.981923 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:41.981884    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:55:57.880773 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:55:57.880689    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:56:11.891972 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:11.891932    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:56:22.079412 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:22.079379    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:56:38.974350 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.974320    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"]
Sep 10 14:56:38.974722 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.974696    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f86912e-43ed-40fe-921e-8a0da7e2d7e7" containerName="istio-proxy"
Sep 10 14:56:38.974722 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.974709    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f86912e-43ed-40fe-921e-8a0da7e2d7e7" containerName="istio-proxy"
Sep 10 14:56:38.974793 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.974782    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f86912e-43ed-40fe-921e-8a0da7e2d7e7" containerName="istio-proxy"
Sep 10 14:56:38.977701 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.977682    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:38.981504 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.981479    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-bd601b\"/\"noten-model-bd601b-kserve-self-signed-certs\""
Sep 10 14:56:38.981641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.981484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-bd601b\"/\"default-dockercfg-m2nxw\""
Sep 10 14:56:38.981641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.981484    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-bd601b\"/\"openshift-service-ca.crt\""
Sep 10 14:56:38.981641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.981492    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-bd601b\"/\"kube-root-ca.crt\""
Sep 10 14:56:38.988064 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:38.988037    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"]
Sep 10 14:56:39.128729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.128696    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwgzq\" (UniqueName: \"kubernetes.io/projected/1c9633b4-e2e7-4fdb-8979-760a6df6e481-kube-api-access-bwgzq\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.128928 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.128740    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-home\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.128928 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.128884    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tmp-dir\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.129047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.128931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tls-certs\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.129047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.128974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-dshm\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.129047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.129007    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-model-cache\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230277 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230162    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tmp-dir\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230277 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230204    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tls-certs\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230277 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230257    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-dshm\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230291    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-model-cache\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwgzq\" (UniqueName: \"kubernetes.io/projected/1c9633b4-e2e7-4fdb-8979-760a6df6e481-kube-api-access-bwgzq\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230389    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-home\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230631    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tmp-dir\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230786 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230744    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-model-cache\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.230786 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.230769    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-home\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.232795 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.232772    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-dshm\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.232949 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.232932    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tls-certs\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.239353 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.239328    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwgzq\" (UniqueName: \"kubernetes.io/projected/1c9633b4-e2e7-4fdb-8979-760a6df6e481-kube-api-access-bwgzq\") pod \"noten-model-bd601b-kserve-5849fc984c-6jjbp\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") " pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.288006 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.287964    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:39.567386 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:39.567358    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"]
Sep 10 14:56:39.569209 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:56:39.569176    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c9633b4_e2e7_4fdb_8979_760a6df6e481.slice/crio-c8204d0e0ee6e0215c2a491469fd773674a12b511d6f15d30207e5dec82bacc9 WatchSource:0}: Error finding container c8204d0e0ee6e0215c2a491469fd773674a12b511d6f15d30207e5dec82bacc9: Status 404 returned error can't find the container with id c8204d0e0ee6e0215c2a491469fd773674a12b511d6f15d30207e5dec82bacc9
Sep 10 14:56:40.559408 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:40.559369    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp" event={"ID":"1c9633b4-e2e7-4fdb-8979-760a6df6e481","Type":"ContainerStarted","Data":"8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3"}
Sep 10 14:56:40.559408 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:40.559411    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp" event={"ID":"1c9633b4-e2e7-4fdb-8979-760a6df6e481","Type":"ContainerStarted","Data":"c8204d0e0ee6e0215c2a491469fd773674a12b511d6f15d30207e5dec82bacc9"}
Sep 10 14:56:40.577524 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:40.577476    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp" podStartSLOduration=2.577460398 podStartE2EDuration="2.577460398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:56:40.57563939 +0000 UTC m=+845.062327305" watchObservedRunningTime="2026-09-10 14:56:40.577460398 +0000 UTC m=+845.064148336"
Sep 10 14:56:49.288692 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:49.288655    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:49.289098 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:49.288716    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:49.301338 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:49.301303    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:49.619432 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:49.619356    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:56:51.177561 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.177532    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"]
Sep 10 14:56:51.181112 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.181097    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.183857 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.183834    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-b56e8eb3-kserve-self-signed-certs\""
Sep 10 14:56:51.189615 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.189591    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:56:51.197904 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.197880    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"]
Sep 10 14:56:51.230069 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.230030    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4befbe-2611-4421-91ef-aa480d346195-tls-certs\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.230290 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.230079    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlddd\" (UniqueName: \"kubernetes.io/projected/4b4befbe-2611-4421-91ef-aa480d346195-kube-api-access-xlddd\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.230290 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.230118    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-dshm\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.230290 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.230138    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-model-cache\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.230290 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.230160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-home\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.230290 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.230272    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-tmp-dir\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331050 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-tmp-dir\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331277 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331092    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4befbe-2611-4421-91ef-aa480d346195-tls-certs\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331277 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331144    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlddd\" (UniqueName: \"kubernetes.io/projected/4b4befbe-2611-4421-91ef-aa480d346195-kube-api-access-xlddd\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331415 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331326    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-dshm\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331415 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331358    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-model-cache\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331415 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331407    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-home\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331635 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331492    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-tmp-dir\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331701 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-model-cache\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.331780 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.331762    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-home\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.333649 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.333626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-dshm\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.333953 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.333935    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4befbe-2611-4421-91ef-aa480d346195-tls-certs\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.339963 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.339938    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlddd\" (UniqueName: \"kubernetes.io/projected/4b4befbe-2611-4421-91ef-aa480d346195-kube-api-access-xlddd\") pod \"e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") " pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.492776 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.492742    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:56:51.768670 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:51.768642    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"]
Sep 10 14:56:51.770673 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:56:51.770646    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b4befbe_2611_4421_91ef_aa480d346195.slice/crio-1910a775e12e21611d05e042c231989e40e89b6484d41011c4c6365a76806667 WatchSource:0}: Error finding container 1910a775e12e21611d05e042c231989e40e89b6484d41011c4c6365a76806667: Status 404 returned error can't find the container with id 1910a775e12e21611d05e042c231989e40e89b6484d41011c4c6365a76806667
Sep 10 14:56:52.633090 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:52.633057    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn" event={"ID":"4b4befbe-2611-4421-91ef-aa480d346195","Type":"ContainerStarted","Data":"6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e"}
Sep 10 14:56:52.633090 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:52.633096    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn" event={"ID":"4b4befbe-2611-4421-91ef-aa480d346195","Type":"ContainerStarted","Data":"1910a775e12e21611d05e042c231989e40e89b6484d41011c4c6365a76806667"}
Sep 10 14:56:52.652471 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:56:52.652408    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn" podStartSLOduration=1.6523867810000001 podStartE2EDuration="1.652386781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:56:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:56:52.652298347 +0000 UTC m=+857.138986240" watchObservedRunningTime="2026-09-10 14:56:52.652386781 +0000 UTC m=+857.139074674"
Sep 10 14:57:01.493599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:01.493555    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:57:01.494101 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:01.493822    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:57:01.506313 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:01.506289    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:57:01.696520 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:01.696495    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:57:06.665642 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:06.665608    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:57:07.183231 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:07.183192    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"]
Sep 10 14:57:07.904787 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:07.904755    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"]
Sep 10 14:57:14.656375 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:14.656287    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"]
Sep 10 14:57:21.067213 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:21.067177    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:57:22.204271 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.204191    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp" podUID="1c9633b4-e2e7-4fdb-8979-760a6df6e481" containerName="main" containerID="cri-o://8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3" gracePeriod=45
Sep 10 14:57:22.448343 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.448317    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:57:22.537719 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.537623    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwgzq\" (UniqueName: \"kubernetes.io/projected/1c9633b4-e2e7-4fdb-8979-760a6df6e481-kube-api-access-bwgzq\") pod \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") "
Sep 10 14:57:22.537719 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.537706    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tls-certs\") pod \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") "
Sep 10 14:57:22.537954 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.537736    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tmp-dir\") pod \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") "
Sep 10 14:57:22.537954 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.537826    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-home\") pod \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") "
Sep 10 14:57:22.537954 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.537860    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-model-cache\") pod \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") "
Sep 10 14:57:22.537954 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.537904    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-dshm\") pod \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\" (UID: \"1c9633b4-e2e7-4fdb-8979-760a6df6e481\") "
Sep 10 14:57:22.538143 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.538020    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-home" (OuterVolumeSpecName: "home") pod "1c9633b4-e2e7-4fdb-8979-760a6df6e481" (UID: "1c9633b4-e2e7-4fdb-8979-760a6df6e481"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:22.538143 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.538049    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1c9633b4-e2e7-4fdb-8979-760a6df6e481" (UID: "1c9633b4-e2e7-4fdb-8979-760a6df6e481"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:22.538143 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.538121    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-model-cache" (OuterVolumeSpecName: "model-cache") pod "1c9633b4-e2e7-4fdb-8979-760a6df6e481" (UID: "1c9633b4-e2e7-4fdb-8979-760a6df6e481"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:22.538291 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.538273    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:22.538291 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.538286    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:22.538401 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.538294    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:22.540135 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.540113    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c9633b4-e2e7-4fdb-8979-760a6df6e481-kube-api-access-bwgzq" (OuterVolumeSpecName: "kube-api-access-bwgzq") pod "1c9633b4-e2e7-4fdb-8979-760a6df6e481" (UID: "1c9633b4-e2e7-4fdb-8979-760a6df6e481"). InnerVolumeSpecName "kube-api-access-bwgzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:57:22.540239 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.540136    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1c9633b4-e2e7-4fdb-8979-760a6df6e481" (UID: "1c9633b4-e2e7-4fdb-8979-760a6df6e481"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:57:22.540239 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.540137    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-dshm" (OuterVolumeSpecName: "dshm") pod "1c9633b4-e2e7-4fdb-8979-760a6df6e481" (UID: "1c9633b4-e2e7-4fdb-8979-760a6df6e481"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:22.638817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.638777    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1c9633b4-e2e7-4fdb-8979-760a6df6e481-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:22.638817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.638809    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwgzq\" (UniqueName: \"kubernetes.io/projected/1c9633b4-e2e7-4fdb-8979-760a6df6e481-kube-api-access-bwgzq\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:22.638817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.638821    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c9633b4-e2e7-4fdb-8979-760a6df6e481-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:22.819389 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.819306    2558 generic.go:356] "Generic (PLEG): container finished" podID="1c9633b4-e2e7-4fdb-8979-760a6df6e481" containerID="8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3" exitCode=0
Sep 10 14:57:22.819389 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.819359    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"
Sep 10 14:57:22.819573 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.819391    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp" event={"ID":"1c9633b4-e2e7-4fdb-8979-760a6df6e481","Type":"ContainerDied","Data":"8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3"}
Sep 10 14:57:22.819573 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.819433    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp" event={"ID":"1c9633b4-e2e7-4fdb-8979-760a6df6e481","Type":"ContainerDied","Data":"c8204d0e0ee6e0215c2a491469fd773674a12b511d6f15d30207e5dec82bacc9"}
Sep 10 14:57:22.819573 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.819449    2558 scope.go:117] "RemoveContainer" containerID="8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3"
Sep 10 14:57:22.831770 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.831748    2558 scope.go:117] "RemoveContainer" containerID="8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3"
Sep 10 14:57:22.832102 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:57:22.832083    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3\": container with ID starting with 8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3 not found: ID does not exist" containerID="8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3"
Sep 10 14:57:22.832171 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.832112    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3"} err="failed to get container status \"8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3\": rpc error: code = NotFound desc = could not find container \"8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3\": container with ID starting with 8b6a0371536371f4b2d1d46e64fa8db4f889b6cde4349d9af947e88fb57577b3 not found: ID does not exist"
Sep 10 14:57:22.860615 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.860582    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"]
Sep 10 14:57:22.868285 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.868263    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-bd601b/noten-model-bd601b-kserve-5849fc984c-6jjbp"]
Sep 10 14:57:22.925051 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:22.925009    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn" podUID="4b4befbe-2611-4421-91ef-aa480d346195" containerName="main" containerID="cri-o://6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e" gracePeriod=45
Sep 10 14:57:23.167758 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.167735    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:57:23.343789 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.343753    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-model-cache\") pod \"4b4befbe-2611-4421-91ef-aa480d346195\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") "
Sep 10 14:57:23.344297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.343821    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4befbe-2611-4421-91ef-aa480d346195-tls-certs\") pod \"4b4befbe-2611-4421-91ef-aa480d346195\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") "
Sep 10 14:57:23.344297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.343855    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlddd\" (UniqueName: \"kubernetes.io/projected/4b4befbe-2611-4421-91ef-aa480d346195-kube-api-access-xlddd\") pod \"4b4befbe-2611-4421-91ef-aa480d346195\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") "
Sep 10 14:57:23.344297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.343926    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-tmp-dir\") pod \"4b4befbe-2611-4421-91ef-aa480d346195\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") "
Sep 10 14:57:23.344297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.343949    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-dshm\") pod \"4b4befbe-2611-4421-91ef-aa480d346195\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") "
Sep 10 14:57:23.344297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.344024    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-home\") pod \"4b4befbe-2611-4421-91ef-aa480d346195\" (UID: \"4b4befbe-2611-4421-91ef-aa480d346195\") "
Sep 10 14:57:23.344297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.344072    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b4befbe-2611-4421-91ef-aa480d346195" (UID: "4b4befbe-2611-4421-91ef-aa480d346195"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:23.344297 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.344181    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b4befbe-2611-4421-91ef-aa480d346195" (UID: "4b4befbe-2611-4421-91ef-aa480d346195"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:23.344620 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.344309    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-home" (OuterVolumeSpecName: "home") pod "4b4befbe-2611-4421-91ef-aa480d346195" (UID: "4b4befbe-2611-4421-91ef-aa480d346195"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:23.344620 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.344353    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:23.344620 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.344374    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:23.346316 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.346288    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b4befbe-2611-4421-91ef-aa480d346195-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b4befbe-2611-4421-91ef-aa480d346195" (UID: "4b4befbe-2611-4421-91ef-aa480d346195"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:57:23.346400 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.346330    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4befbe-2611-4421-91ef-aa480d346195-kube-api-access-xlddd" (OuterVolumeSpecName: "kube-api-access-xlddd") pod "4b4befbe-2611-4421-91ef-aa480d346195" (UID: "4b4befbe-2611-4421-91ef-aa480d346195"). InnerVolumeSpecName "kube-api-access-xlddd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:57:23.346400 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.346333    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-dshm" (OuterVolumeSpecName: "dshm") pod "4b4befbe-2611-4421-91ef-aa480d346195" (UID: "4b4befbe-2611-4421-91ef-aa480d346195"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:23.445496 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.445458    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:23.445496 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.445488    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4befbe-2611-4421-91ef-aa480d346195-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:23.445496 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.445497    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4befbe-2611-4421-91ef-aa480d346195-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:23.445722 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.445507    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlddd\" (UniqueName: \"kubernetes.io/projected/4b4befbe-2611-4421-91ef-aa480d346195-kube-api-access-xlddd\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:23.828244 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.828183    2558 generic.go:356] "Generic (PLEG): container finished" podID="4b4befbe-2611-4421-91ef-aa480d346195" containerID="6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e" exitCode=0
Sep 10 14:57:23.828244 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.828244    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"
Sep 10 14:57:23.828501 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.828262    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn" event={"ID":"4b4befbe-2611-4421-91ef-aa480d346195","Type":"ContainerDied","Data":"6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e"}
Sep 10 14:57:23.828501 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.828300    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn" event={"ID":"4b4befbe-2611-4421-91ef-aa480d346195","Type":"ContainerDied","Data":"1910a775e12e21611d05e042c231989e40e89b6484d41011c4c6365a76806667"}
Sep 10 14:57:23.828501 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.828321    2558 scope.go:117] "RemoveContainer" containerID="6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e"
Sep 10 14:57:23.837843 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.837822    2558 scope.go:117] "RemoveContainer" containerID="6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e"
Sep 10 14:57:23.838153 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:57:23.838134    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e\": container with ID starting with 6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e not found: ID does not exist" containerID="6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e"
Sep 10 14:57:23.838248 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.838163    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e"} err="failed to get container status \"6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e\": rpc error: code = NotFound desc = could not find container \"6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e\": container with ID starting with 6fa98c592701282158652f1f58ee2a37690b36f92e10ad471fa4537d478b1d8e not found: ID does not exist"
Sep 10 14:57:23.850415 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.850388    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"]
Sep 10 14:57:23.852760 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:23.852740    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-b56e8eb3-kserve-cc6f94d75-nfwwn"]
Sep 10 14:57:24.019918 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:24.019884    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c9633b4-e2e7-4fdb-8979-760a6df6e481" path="/var/lib/kubelet/pods/1c9633b4-e2e7-4fdb-8979-760a6df6e481/volumes"
Sep 10 14:57:24.020304 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:24.020291    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b4befbe-2611-4421-91ef-aa480d346195" path="/var/lib/kubelet/pods/4b4befbe-2611-4421-91ef-aa480d346195/volumes"
Sep 10 14:57:29.676693 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:29.676606    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8" podUID="72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" containerName="main" containerID="cri-o://50cc7efe16ad56cdbe128a3267ed2e5f8463bb64d7821bcb746b5e4d64becfc4" gracePeriod=45
Sep 10 14:57:29.866456 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:29.866415    2558 generic.go:356] "Generic (PLEG): container finished" podID="72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" containerID="50cc7efe16ad56cdbe128a3267ed2e5f8463bb64d7821bcb746b5e4d64becfc4" exitCode=0
Sep 10 14:57:29.866456 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:29.866460    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8" event={"ID":"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b","Type":"ContainerDied","Data":"50cc7efe16ad56cdbe128a3267ed2e5f8463bb64d7821bcb746b5e4d64becfc4"}
Sep 10 14:57:29.923714 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:29.923690    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:57:30.010339 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010306    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-home\") pod \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") "
Sep 10 14:57:30.010510 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010363    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tmp-dir\") pod \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") "
Sep 10 14:57:30.010510 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010385    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9tvz\" (UniqueName: \"kubernetes.io/projected/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-kube-api-access-m9tvz\") pod \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") "
Sep 10 14:57:30.010510 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010420    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-dshm\") pod \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") "
Sep 10 14:57:30.010510 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010479    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-model-cache\") pod \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") "
Sep 10 14:57:30.010510 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010508    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tls-certs\") pod \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\" (UID: \"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b\") "
Sep 10 14:57:30.010776 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010631    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" (UID: "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:30.010776 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010638    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-home" (OuterVolumeSpecName: "home") pod "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" (UID: "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:30.010776 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010760    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:30.010904 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010777    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:30.010904 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.010811    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" (UID: "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:30.012773 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.012746    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-dshm" (OuterVolumeSpecName: "dshm") pod "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" (UID: "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:30.012891 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.012800    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" (UID: "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:57:30.012891 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.012866    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-kube-api-access-m9tvz" (OuterVolumeSpecName: "kube-api-access-m9tvz") pod "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" (UID: "72fdc00e-1d59-45bb-9010-c83cb0d1fb0b"). InnerVolumeSpecName "kube-api-access-m9tvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:57:30.112326 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.112284    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:30.112326 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.112322    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:30.112326 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.112336    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:30.112554 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.112348    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9tvz\" (UniqueName: \"kubernetes.io/projected/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b-kube-api-access-m9tvz\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:57:30.872961 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.872935    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"
Sep 10 14:57:30.873394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.872973    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8" event={"ID":"72fdc00e-1d59-45bb-9010-c83cb0d1fb0b","Type":"ContainerDied","Data":"e4b7be56d0fe5c59204bf9fedec6c32cd26c6a85ed205e139867cf858fc9535a"}
Sep 10 14:57:30.873394 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.873027    2558 scope.go:117] "RemoveContainer" containerID="50cc7efe16ad56cdbe128a3267ed2e5f8463bb64d7821bcb746b5e4d64becfc4"
Sep 10 14:57:30.890853 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.890823    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"]
Sep 10 14:57:30.895086 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:30.895067    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-b56e8eb3-kserve-f9bd5cf7c-mc9z8"]
Sep 10 14:57:32.018581 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:32.018549    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" path="/var/lib/kubelet/pods/72fdc00e-1d59-45bb-9010-c83cb0d1fb0b/volumes"
Sep 10 14:57:34.356355 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356318    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"]
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356681    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b4befbe-2611-4421-91ef-aa480d346195" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356691    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b4befbe-2611-4421-91ef-aa480d346195" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356709    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c9633b4-e2e7-4fdb-8979-760a6df6e481" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356714    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c9633b4-e2e7-4fdb-8979-760a6df6e481" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356724    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356728    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356805    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c9633b4-e2e7-4fdb-8979-760a6df6e481" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356813    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="72fdc00e-1d59-45bb-9010-c83cb0d1fb0b" containerName="main"
Sep 10 14:57:34.356992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.356821    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b4befbe-2611-4421-91ef-aa480d346195" containerName="main"
Sep 10 14:57:34.359992 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.359971    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.362446 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.362421    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:57:34.362446 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.362426    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-22282287-openshift-default-dockercfg-f4cf9\""
Sep 10 14:57:34.372488 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.372461    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"]
Sep 10 14:57:34.453350 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453313    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-istio-token\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453379    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6948579-adcf-47f6-b463-fb4742bf433e-istio-podinfo\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453450    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-credential-socket\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453494    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-certs\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453542 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453512    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6948579-adcf-47f6-b463-fb4742bf433e-istiod-ca-cert\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453745 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453562    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-data\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453745 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-socket\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453745 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453705    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8k6l\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-kube-api-access-k8k6l\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.453745 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.453728    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-envoy\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555189 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555152    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-socket\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555189 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555192    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8k6l\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-kube-api-access-k8k6l\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555444 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555209    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-envoy\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555444 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555369    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-istio-token\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555570 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6948579-adcf-47f6-b463-fb4742bf433e-istio-podinfo\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555570 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555500    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-credential-socket\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555570 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555540    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-certs\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555570 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555565    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6948579-adcf-47f6-b463-fb4742bf433e-istiod-ca-cert\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555591    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-socket\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555598    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-data\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555917 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555854    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-credential-socket\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.555917 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555875    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-data\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.556028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.555920    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-certs\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.556325 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.556302    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6948579-adcf-47f6-b463-fb4742bf433e-istiod-ca-cert\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.557763 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.557741    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-envoy\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.557939 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.557923    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6948579-adcf-47f6-b463-fb4742bf433e-istio-podinfo\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.563125 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.563100    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8k6l\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-kube-api-access-k8k6l\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.563500 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.563479    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-istio-token\") pod \"e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") " pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.670408 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.670314    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:34.769131 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.769099    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"]
Sep 10 14:57:34.772899 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.772882    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.776114 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.776089    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ff661a\"/\"default-dockercfg-2z49f\""
Sep 10 14:57:34.776285 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.776142    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ff661a\"/\"openshift-service-ca.crt\""
Sep 10 14:57:34.776285 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.776203    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ff661a\"/\"test-model-ff661a-kserve-self-signed-certs\""
Sep 10 14:57:34.777184 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.777168    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ff661a\"/\"kube-root-ca.crt\""
Sep 10 14:57:34.784426 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.784395    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"]
Sep 10 14:57:34.858920 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.858888    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-dshm\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.859083 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.858944    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jzvf\" (UniqueName: \"kubernetes.io/projected/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-kube-api-access-2jzvf\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.859083 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.859068    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tmp-dir\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.859176 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.859103    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-home\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.859176 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.859127    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-model-cache\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.859176 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.859146    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tls-certs\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.952482 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.952456    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"]
Sep 10 14:57:34.954336 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:57:34.954305    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6948579_adcf_47f6_b463_fb4742bf433e.slice/crio-273ffbd8abcb15e489eb08fce040daf381fb7b95661b8fe3548b94e4144f60e0 WatchSource:0}: Error finding container 273ffbd8abcb15e489eb08fce040daf381fb7b95661b8fe3548b94e4144f60e0: Status 404 returned error can't find the container with id 273ffbd8abcb15e489eb08fce040daf381fb7b95661b8fe3548b94e4144f60e0
Sep 10 14:57:34.956574 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.956539    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:57:34.956674 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.956621    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:57:34.956674 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.956662    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:57:34.959988 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.959968    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-dshm\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.960069 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.960001    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jzvf\" (UniqueName: \"kubernetes.io/projected/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-kube-api-access-2jzvf\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.960069 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.960043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tmp-dir\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.960069 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.960061    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-home\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.960381 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.960078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-model-cache\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.960381 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.960092    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tls-certs\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.961642 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.960842    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tmp-dir\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.961642 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.961081    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-model-cache\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.961642 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.961595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-home\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.962766 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.962750    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-dshm\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.963294 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.963204    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tls-certs\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:34.971618 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:34.971590    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jzvf\" (UniqueName: \"kubernetes.io/projected/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-kube-api-access-2jzvf\") pod \"test-model-ff661a-kserve-56b944c5bf-g944f\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") " pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:35.084189 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.084152    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:35.362964 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.362940    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"]
Sep 10 14:57:35.364965 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:57:35.364938    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaff6ff41_0d66_42b3_a3e1_4d59f640ec3b.slice/crio-84bb3d8b1c3f293e427d5344f65da93070935465714172e1c4887b3f721dd6b6 WatchSource:0}: Error finding container 84bb3d8b1c3f293e427d5344f65da93070935465714172e1c4887b3f721dd6b6: Status 404 returned error can't find the container with id 84bb3d8b1c3f293e427d5344f65da93070935465714172e1c4887b3f721dd6b6
Sep 10 14:57:35.906525 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.906489    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" event={"ID":"d6948579-adcf-47f6-b463-fb4742bf433e","Type":"ContainerStarted","Data":"d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b"}
Sep 10 14:57:35.906730 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.906558    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" event={"ID":"d6948579-adcf-47f6-b463-fb4742bf433e","Type":"ContainerStarted","Data":"273ffbd8abcb15e489eb08fce040daf381fb7b95661b8fe3548b94e4144f60e0"}
Sep 10 14:57:35.907971 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.907936    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f" event={"ID":"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b","Type":"ContainerStarted","Data":"fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61"}
Sep 10 14:57:35.908071 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.907972    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f" event={"ID":"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b","Type":"ContainerStarted","Data":"84bb3d8b1c3f293e427d5344f65da93070935465714172e1c4887b3f721dd6b6"}
Sep 10 14:57:35.931811 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.931759    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" podStartSLOduration=1.93174311 podStartE2EDuration="1.93174311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:57:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:57:35.929294018 +0000 UTC m=+900.415981922" watchObservedRunningTime="2026-09-10 14:57:35.93174311 +0000 UTC m=+900.418431001"
Sep 10 14:57:35.949769 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:35.949715    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f" podStartSLOduration=1.949695202 podStartE2EDuration="1.949695202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:57:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:57:35.946724095 +0000 UTC m=+900.433411988" watchObservedRunningTime="2026-09-10 14:57:35.949695202 +0000 UTC m=+900.436383096"
Sep 10 14:57:36.670689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:36.670655    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:36.675522 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:36.675498    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:36.913995 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:36.913966    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:36.914889 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:36.914868    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:57:37.377558 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:37.377522    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:57:45.084973 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:45.084938    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:45.087416 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:45.084996    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:45.097590 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:45.097571    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:45.977800 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:45.977767    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:57:55.779668 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:55.779628    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:57:59.046959 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:59.046924    2558 generic.go:356] "Generic (PLEG): container finished" podID="d6948579-adcf-47f6-b463-fb4742bf433e" containerID="d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b" exitCode=0
Sep 10 14:57:59.047408 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:59.047004    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" event={"ID":"d6948579-adcf-47f6-b463-fb4742bf433e","Type":"ContainerDied","Data":"d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b"}
Sep 10 14:57:59.047468 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:59.047421    2558 scope.go:117] "RemoveContainer" containerID="d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b"
Sep 10 14:57:59.048564 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:59.048535    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:57:59.048627 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:59.048604    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:57:59.048663 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:57:59.048636    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:58:00.054921 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:00.054888    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" event={"ID":"d6948579-adcf-47f6-b463-fb4742bf433e","Type":"ContainerStarted","Data":"17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb"}
Sep 10 14:58:00.671299 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:00.671257    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:58:00.671606 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:00.671580    2558 patch_prober.go:28] interesting pod/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 10 14:58:00.671696 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:00.671644    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 10 14:58:01.671695 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:01.671653    2558 patch_prober.go:28] interesting pod/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 10 14:58:01.672169 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:01.671734    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 10 14:58:02.670924 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:02.670883    2558 patch_prober.go:28] interesting pod/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 10 14:58:02.671100 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:02.670943    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 10 14:58:03.671614 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:03.671576    2558 patch_prober.go:28] interesting pod/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 10 14:58:03.672003 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:03.671649    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 10 14:58:04.670861 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:04.670816    2558 patch_prober.go:28] interesting pod/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 10 14:58:04.671042 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:04.670885    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 10 14:58:05.675129 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:05.675095    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:58:05.675664 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:05.675441    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:58:05.676117 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:05.676097    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:58:10.118651 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.118617    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"]
Sep 10 14:58:10.122156 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.122140    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"
Sep 10 14:58:10.124797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.124776    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 14:58:10.124797 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.124790    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:58:10.130683 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.130659    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"]
Sep 10 14:58:10.200850 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.200818    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/483e8d01-8f04-487f-ae31-c4930610f8a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-22282287-dt6bx\" (UID: \"483e8d01-8f04-487f-ae31-c4930610f8a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"
Sep 10 14:58:10.301673 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.301639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/483e8d01-8f04-487f-ae31-c4930610f8a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-22282287-dt6bx\" (UID: \"483e8d01-8f04-487f-ae31-c4930610f8a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"
Sep 10 14:58:10.302425 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.302402    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/483e8d01-8f04-487f-ae31-c4930610f8a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-22282287-dt6bx\" (UID: \"483e8d01-8f04-487f-ae31-c4930610f8a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"
Sep 10 14:58:10.369886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.369796    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:58:10.433759 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.433723    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"
Sep 10 14:58:10.707688 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:10.707665    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"]
Sep 10 14:58:10.709360 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:58:10.709334    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483e8d01_8f04_487f_ae31_c4930610f8a5.slice/crio-a34a0c3da2819ac66cced6743a86dc03d5a4a6aebdeb99e277afae988091b2ff WatchSource:0}: Error finding container a34a0c3da2819ac66cced6743a86dc03d5a4a6aebdeb99e277afae988091b2ff: Status 404 returned error can't find the container with id a34a0c3da2819ac66cced6743a86dc03d5a4a6aebdeb99e277afae988091b2ff
Sep 10 14:58:11.118886 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:11.118801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx" event={"ID":"483e8d01-8f04-487f-ae31-c4930610f8a5","Type":"ContainerStarted","Data":"a34a0c3da2819ac66cced6743a86dc03d5a4a6aebdeb99e277afae988091b2ff"}
Sep 10 14:58:14.143900 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:14.143859    2558 generic.go:356] "Generic (PLEG): container finished" podID="483e8d01-8f04-487f-ae31-c4930610f8a5" containerID="557aa338de0e0b66e31fd2bc50ffbec5cf3d3cc567bc318d0108eb45ae5dcd51" exitCode=0
Sep 10 14:58:14.144276 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:14.143934    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx" event={"ID":"483e8d01-8f04-487f-ae31-c4930610f8a5","Type":"ContainerDied","Data":"557aa338de0e0b66e31fd2bc50ffbec5cf3d3cc567bc318d0108eb45ae5dcd51"}
Sep 10 14:58:15.283451 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:15.283427    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"
Sep 10 14:58:15.348080 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:15.348053    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/483e8d01-8f04-487f-ae31-c4930610f8a5-maas-api-service-ca\") pod \"483e8d01-8f04-487f-ae31-c4930610f8a5\" (UID: \"483e8d01-8f04-487f-ae31-c4930610f8a5\") "
Sep 10 14:58:15.348439 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:15.348416    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/483e8d01-8f04-487f-ae31-c4930610f8a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "483e8d01-8f04-487f-ae31-c4930610f8a5" (UID: "483e8d01-8f04-487f-ae31-c4930610f8a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:58:15.449170 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:15.449142    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/483e8d01-8f04-487f-ae31-c4930610f8a5-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:16.155862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:16.155826    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx" event={"ID":"483e8d01-8f04-487f-ae31-c4930610f8a5","Type":"ContainerDied","Data":"a34a0c3da2819ac66cced6743a86dc03d5a4a6aebdeb99e277afae988091b2ff"}
Sep 10 14:58:16.156031 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:16.155888    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a34a0c3da2819ac66cced6743a86dc03d5a4a6aebdeb99e277afae988091b2ff"
Sep 10 14:58:16.156031 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:16.155843    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"
Sep 10 14:58:20.845558 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:20.845480    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"]
Sep 10 14:58:20.848333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:20.848304    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-22282287-dt6bx"]
Sep 10 14:58:22.019538 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:22.019503    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="483e8d01-8f04-487f-ae31-c4930610f8a5" path="/var/lib/kubelet/pods/483e8d01-8f04-487f-ae31-c4930610f8a5/volumes"
Sep 10 14:58:24.774598 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:24.774565    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:58:27.357566 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:27.357530    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"]
Sep 10 14:58:27.358034 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:27.357758    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy" containerID="cri-o://17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb" gracePeriod=30
Sep 10 14:58:32.601502 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.601473    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:58:32.715700 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715669    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6948579-adcf-47f6-b463-fb4742bf433e-istio-podinfo\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.715863 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715707    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-socket\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.715863 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715723    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-envoy\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.715863 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715745    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-certs\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.715863 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715839    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8k6l\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-kube-api-access-k8k6l\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.716070 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715880    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6948579-adcf-47f6-b463-fb4742bf433e-istiod-ca-cert\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.716070 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715900    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-data\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.716070 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715946    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-credential-socket\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.716070 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.715971    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:32.716070 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716065    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-istio-token\") pod \"d6948579-adcf-47f6-b463-fb4742bf433e\" (UID: \"d6948579-adcf-47f6-b463-fb4742bf433e\") "
Sep 10 14:58:32.716375 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716172    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:32.716375 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716185    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:32.716489 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716438    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6948579-adcf-47f6-b463-fb4742bf433e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:58:32.716546 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716504    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:32.716689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716574    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.716689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716663    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-data\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.716689 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.716676    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-credential-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.718112 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.718089    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:58:32.718291 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.718263    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:58:32.718586 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.718567    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6948579-adcf-47f6-b463-fb4742bf433e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:58:32.718643 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.718625    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-kube-api-access-k8k6l" (OuterVolumeSpecName: "kube-api-access-k8k6l") pod "d6948579-adcf-47f6-b463-fb4742bf433e" (UID: "d6948579-adcf-47f6-b463-fb4742bf433e"). InnerVolumeSpecName "kube-api-access-k8k6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:58:32.817815 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.817784    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-istio-token\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.817815 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.817808    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6948579-adcf-47f6-b463-fb4742bf433e-istio-podinfo\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.817999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.817825    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-istio-envoy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.817999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.817834    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6948579-adcf-47f6-b463-fb4742bf433e-workload-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.817999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.817842    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8k6l\" (UniqueName: \"kubernetes.io/projected/d6948579-adcf-47f6-b463-fb4742bf433e-kube-api-access-k8k6l\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:32.817999 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:32.817850    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6948579-adcf-47f6-b463-fb4742bf433e-istiod-ca-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:58:33.248374 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.248335    2558 generic.go:356] "Generic (PLEG): container finished" podID="d6948579-adcf-47f6-b463-fb4742bf433e" containerID="17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb" exitCode=0
Sep 10 14:58:33.248567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.248382    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"
Sep 10 14:58:33.248567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.248413    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" event={"ID":"d6948579-adcf-47f6-b463-fb4742bf433e","Type":"ContainerDied","Data":"17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb"}
Sep 10 14:58:33.248567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.248465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m" event={"ID":"d6948579-adcf-47f6-b463-fb4742bf433e","Type":"ContainerDied","Data":"273ffbd8abcb15e489eb08fce040daf381fb7b95661b8fe3548b94e4144f60e0"}
Sep 10 14:58:33.248567 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.248486    2558 scope.go:117] "RemoveContainer" containerID="17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb"
Sep 10 14:58:33.261110 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.261082    2558 scope.go:117] "RemoveContainer" containerID="d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b"
Sep 10 14:58:33.269822 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.269796    2558 scope.go:117] "RemoveContainer" containerID="17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb"
Sep 10 14:58:33.270122 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:58:33.270102    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb\": container with ID starting with 17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb not found: ID does not exist" containerID="17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb"
Sep 10 14:58:33.270203 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.270136    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb"} err="failed to get container status \"17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb\": rpc error: code = NotFound desc = could not find container \"17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb\": container with ID starting with 17068059eb0d80c87c825a8c580f970559805faa78b77cd2463103d4d6a99ebb not found: ID does not exist"
Sep 10 14:58:33.270203 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.270162    2558 scope.go:117] "RemoveContainer" containerID="d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b"
Sep 10 14:58:33.270580 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:58:33.270562    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b\": container with ID starting with d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b not found: ID does not exist" containerID="d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b"
Sep 10 14:58:33.270627 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.270588    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b"} err="failed to get container status \"d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b\": rpc error: code = NotFound desc = could not find container \"d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b\": container with ID starting with d8b05234e59639028b2d1f06aaf0cefcfaaeb1d574410bd7593df7fe14cf0d5b not found: ID does not exist"
Sep 10 14:58:33.274082 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.274061    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"]
Sep 10 14:58:33.278792 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:33.278770    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-22282287-openshift-default-7f4865bc48-hxl2m"]
Sep 10 14:58:34.019619 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:34.019586    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" path="/var/lib/kubelet/pods/d6948579-adcf-47f6-b463-fb4742bf433e/volumes"
Sep 10 14:58:37.167823 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:37.167788    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:58:40.963796 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.963765    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"]
Sep 10 14:58:40.964272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.964122    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy"
Sep 10 14:58:40.964272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.964133    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy"
Sep 10 14:58:40.964272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.964149    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483e8d01-8f04-487f-ae31-c4930610f8a5" containerName="revoke-keys"
Sep 10 14:58:40.964272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.964155    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="483e8d01-8f04-487f-ae31-c4930610f8a5" containerName="revoke-keys"
Sep 10 14:58:40.964272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.964245    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="483e8d01-8f04-487f-ae31-c4930610f8a5" containerName="revoke-keys"
Sep 10 14:58:40.964272 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.964273    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy"
Sep 10 14:58:40.964604 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.964281    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy"
Sep 10 14:58:40.970188 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.970166    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:40.973882 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.973847    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:58:40.974006 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.973963    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-5fb994-openshift-default-dockercfg-gw8g9\""
Sep 10 14:58:40.979315 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:40.979290    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"]
Sep 10 14:58:41.096556 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-credential-socket\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096736 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096572    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-data\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096736 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096632    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-istio-token\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096736 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-envoy\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096736 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096722    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2dd132-fabe-4915-a362-acc938885e52-istiod-ca-cert\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096946 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096769    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-certs\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096946 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096784    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2dd132-fabe-4915-a362-acc938885e52-istio-podinfo\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096946 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096856    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blh5n\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-kube-api-access-blh5n\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.096946 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.096894    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-socket\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-certs\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197459    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2dd132-fabe-4915-a362-acc938885e52-istio-podinfo\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blh5n\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-kube-api-access-blh5n\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-socket\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197595    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-credential-socket\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197636    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-data\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197675    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-istio-token\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.197717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197709    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-envoy\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.198047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197748    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2dd132-fabe-4915-a362-acc938885e52-istiod-ca-cert\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.198047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197877    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-certs\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.198047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.197933    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-socket\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.198047 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.198043    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-credential-socket\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.198280 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.198131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-data\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.198455 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.198436    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2dd132-fabe-4915-a362-acc938885e52-istiod-ca-cert\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.204615 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.200708    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2dd132-fabe-4915-a362-acc938885e52-istio-podinfo\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.204615 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.204284    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-envoy\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.207552 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.206428    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-istio-token\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.209584 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.209557    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blh5n\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-kube-api-access-blh5n\") pod \"e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") " pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.281282 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.281196    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:41.561657 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.561630    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"]
Sep 10 14:58:41.562938 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:58:41.562913    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c2dd132_fabe_4915_a362_acc938885e52.slice/crio-7ef1d699a1e13c3a795d73380d71b2abb887cac72e2c11ba7129a1ca0bf3c76c WatchSource:0}: Error finding container 7ef1d699a1e13c3a795d73380d71b2abb887cac72e2c11ba7129a1ca0bf3c76c: Status 404 returned error can't find the container with id 7ef1d699a1e13c3a795d73380d71b2abb887cac72e2c11ba7129a1ca0bf3c76c
Sep 10 14:58:41.565167 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.565133    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:58:41.565241 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.565204    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:58:41.565288 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:41.565251    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:58:42.300431 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:42.300394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" event={"ID":"9c2dd132-fabe-4915-a362-acc938885e52","Type":"ContainerStarted","Data":"b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617"}
Sep 10 14:58:42.300812 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:42.300459    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" event={"ID":"9c2dd132-fabe-4915-a362-acc938885e52","Type":"ContainerStarted","Data":"7ef1d699a1e13c3a795d73380d71b2abb887cac72e2c11ba7129a1ca0bf3c76c"}
Sep 10 14:58:42.324134 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:42.324086    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" podStartSLOduration=2.324069418 podStartE2EDuration="2.324069418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:58:42.321858128 +0000 UTC m=+966.808546022" watchObservedRunningTime="2026-09-10 14:58:42.324069418 +0000 UTC m=+966.810757310"
Sep 10 14:58:43.282037 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:43.281996    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:43.287030 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:43.287003    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:43.306517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:43.306491    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:43.307517 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:43.307500    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:58:53.172246 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:53.172192    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:58:56.063641 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.063602    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"]
Sep 10 14:58:56.193573 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.193536    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"]
Sep 10 14:58:56.194139 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.194113    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy"
Sep 10 14:58:56.194139 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.194141    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6948579-adcf-47f6-b463-fb4742bf433e" containerName="istio-proxy"
Sep 10 14:58:56.200572 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.200537    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"
Sep 10 14:58:56.204777 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.204744    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"]
Sep 10 14:58:56.205707 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.205678    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 14:58:56.205958 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.205937    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:58:56.243484 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.243453    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc\" (UID: \"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"
Sep 10 14:58:56.344656 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.344558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc\" (UID: \"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"
Sep 10 14:58:56.345171 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.345150    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc\" (UID: \"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"
Sep 10 14:58:56.517637 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.517601    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"
Sep 10 14:58:56.796739 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:56.796689    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"]
Sep 10 14:58:57.387375 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:57.387235    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc" event={"ID":"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9","Type":"ContainerStarted","Data":"1b00d991271bbac9bbe05b4c3ef2849653396d1e3e6810b5140cdf1321546d77"}
Sep 10 14:58:57.387375 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:57.387311    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc" event={"ID":"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9","Type":"ContainerStarted","Data":"3fa1175207f752fb6f648c7b9b174235cc6e42f2820933704ca7a1234736b5fe"}
Sep 10 14:58:57.407554 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:57.407505    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc" podStartSLOduration=1.407487695 podStartE2EDuration="1.407487695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:58:57.405945436 +0000 UTC m=+981.892633328" watchObservedRunningTime="2026-09-10 14:58:57.407487695 +0000 UTC m=+981.894175585"
Sep 10 14:58:58.401691 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:58.401655    2558 generic.go:356] "Generic (PLEG): container finished" podID="ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9" containerID="1b00d991271bbac9bbe05b4c3ef2849653396d1e3e6810b5140cdf1321546d77" exitCode=0
Sep 10 14:58:58.402393 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:58.402356    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc" event={"ID":"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9","Type":"ContainerDied","Data":"1b00d991271bbac9bbe05b4c3ef2849653396d1e3e6810b5140cdf1321546d77"}
Sep 10 14:58:59.559372 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:59.559343    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"
Sep 10 14:58:59.676706 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:59.676623    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9-maas-api-service-ca\") pod \"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9\" (UID: \"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9\") "
Sep 10 14:58:59.676959 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:59.676935    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9" (UID: "ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:58:59.778021 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:58:59.777980    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:00.415906 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:00.415811    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"
Sep 10 14:59:00.415906 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:00.415838    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc" event={"ID":"ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9","Type":"ContainerDied","Data":"3fa1175207f752fb6f648c7b9b174235cc6e42f2820933704ca7a1234736b5fe"}
Sep 10 14:59:00.415906 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:00.415900    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fa1175207f752fb6f648c7b9b174235cc6e42f2820933704ca7a1234736b5fe"
Sep 10 14:59:03.435446 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:03.435410    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c2dd132-fabe-4915-a362-acc938885e52" containerID="b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617" exitCode=0
Sep 10 14:59:03.435885 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:03.435486    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" event={"ID":"9c2dd132-fabe-4915-a362-acc938885e52","Type":"ContainerDied","Data":"b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617"}
Sep 10 14:59:03.435885 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:03.435881    2558 scope.go:117] "RemoveContainer" containerID="b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617"
Sep 10 14:59:03.436975 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:03.436944    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:03.437059 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:03.437011    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:03.437059 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:03.437040    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:04.443103 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:04.443066    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" event={"ID":"9c2dd132-fabe-4915-a362-acc938885e52","Type":"ContainerStarted","Data":"f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115"}
Sep 10 14:59:05.282564 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:05.281764    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:59:05.282564 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:05.282182    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:59:05.282564 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:05.282259    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:59:06.293246 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:06.282676    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:59:06.293246 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:06.282742    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:59:06.565951 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:06.565870    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:59:07.282333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:07.282286    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:59:07.282740 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:07.282708    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:59:07.753748 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:07.753708    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"]
Sep 10 14:59:07.756726 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:07.756701    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-74eee9-67zhc"]
Sep 10 14:59:08.019767 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:08.019693    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9" path="/var/lib/kubelet/pods/ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9/volumes"
Sep 10 14:59:08.282723 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:08.282619    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:59:08.282723 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:08.282688    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:59:09.282345 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:09.282303    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:59:09.282862 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:09.282376    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:59:10.285588 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:10.285558    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:59:10.286049 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:10.285865    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:59:10.286693 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:10.286676    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 14:59:11.107855 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.107811    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f" podUID="aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" containerName="main" containerID="cri-o://fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61" gracePeriod=45
Sep 10 14:59:11.362182 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.362119    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:59:11.485913 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.485874    2558 generic.go:356] "Generic (PLEG): container finished" podID="aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" containerID="fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61" exitCode=0
Sep 10 14:59:11.486082 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.485920    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"
Sep 10 14:59:11.486082 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.485961    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f" event={"ID":"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b","Type":"ContainerDied","Data":"fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61"}
Sep 10 14:59:11.486082 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.486001    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f" event={"ID":"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b","Type":"ContainerDied","Data":"84bb3d8b1c3f293e427d5344f65da93070935465714172e1c4887b3f721dd6b6"}
Sep 10 14:59:11.486082 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.486027    2558 scope.go:117] "RemoveContainer" containerID="fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61"
Sep 10 14:59:11.491043 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491017    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-dshm\") pod \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") "
Sep 10 14:59:11.491149 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491073    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-model-cache\") pod \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") "
Sep 10 14:59:11.491149 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491097    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tmp-dir\") pod \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") "
Sep 10 14:59:11.491149 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491126    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-home\") pod \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") "
Sep 10 14:59:11.491427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491257    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tls-certs\") pod \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") "
Sep 10 14:59:11.491427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491279    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jzvf\" (UniqueName: \"kubernetes.io/projected/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-kube-api-access-2jzvf\") pod \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\" (UID: \"aff6ff41-0d66-42b3-a3e1-4d59f640ec3b\") "
Sep 10 14:59:11.491427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491386    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-model-cache" (OuterVolumeSpecName: "model-cache") pod "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" (UID: "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:59:11.491427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491405    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" (UID: "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:59:11.491598 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491453    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-home" (OuterVolumeSpecName: "home") pod "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" (UID: "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:59:11.491688 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491630    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:11.491820 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491740    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:11.491884 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.491822    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:11.493357 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.493332    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-dshm" (OuterVolumeSpecName: "dshm") pod "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" (UID: "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:59:11.493515 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.493494    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-kube-api-access-2jzvf" (OuterVolumeSpecName: "kube-api-access-2jzvf") pod "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" (UID: "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b"). InnerVolumeSpecName "kube-api-access-2jzvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:59:11.493578 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.493555    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" (UID: "aff6ff41-0d66-42b3-a3e1-4d59f640ec3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:59:11.496173 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.496155    2558 scope.go:117] "RemoveContainer" containerID="fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61"
Sep 10 14:59:11.496540 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:11.496520    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61\": container with ID starting with fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61 not found: ID does not exist" containerID="fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61"
Sep 10 14:59:11.496630 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.496550    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61"} err="failed to get container status \"fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61\": rpc error: code = NotFound desc = could not find container \"fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61\": container with ID starting with fd143c04479bfdc3a3d0facf778406493fcde37996d570d6f18b3300ecb3ff61 not found: ID does not exist"
Sep 10 14:59:11.593365 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.593328    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:11.593365 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.593364    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jzvf\" (UniqueName: \"kubernetes.io/projected/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-kube-api-access-2jzvf\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:11.593577 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.593376    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:11.809271 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.809209    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"]
Sep 10 14:59:11.811532 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:11.811506    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ff661a/test-model-ff661a-kserve-56b944c5bf-g944f"]
Sep 10 14:59:12.019118 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:12.019085    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" path="/var/lib/kubelet/pods/aff6ff41-0d66-42b3-a3e1-4d59f640ec3b/volumes"
Sep 10 14:59:15.342676 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.342629    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"]
Sep 10 14:59:15.343257 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.343201    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9" containerName="revoke-keys"
Sep 10 14:59:15.343257 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.343241    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9" containerName="revoke-keys"
Sep 10 14:59:15.343416 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.343261    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" containerName="main"
Sep 10 14:59:15.343416 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.343269    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" containerName="main"
Sep 10 14:59:15.343416 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.343391    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea40e70a-6aa5-4c9e-a13c-61eeb9cca2b9" containerName="revoke-keys"
Sep 10 14:59:15.343416 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.343411    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff6ff41-0d66-42b3-a3e1-4d59f640ec3b" containerName="main"
Sep 10 14:59:15.348119 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.348095    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"
Sep 10 14:59:15.350636 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.350604    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:59:15.351970 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.351840    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 14:59:15.354374 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.354351    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"]
Sep 10 14:59:15.429029 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.428991    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ee203-8230-413a-8455-33757443992b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm\" (UID: \"775ee203-8230-413a-8455-33757443992b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"
Sep 10 14:59:15.529619 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.529590    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ee203-8230-413a-8455-33757443992b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm\" (UID: \"775ee203-8230-413a-8455-33757443992b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"
Sep 10 14:59:15.530208 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.530189    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ee203-8230-413a-8455-33757443992b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm\" (UID: \"775ee203-8230-413a-8455-33757443992b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"
Sep 10 14:59:15.575324 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.575291    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:59:15.660613 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.660534    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"
Sep 10 14:59:15.931976 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:15.931948    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"]
Sep 10 14:59:15.934748 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:59:15.934718    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775ee203_8230_413a_8455_33757443992b.slice/crio-936f664e788f1e699697dceb9e4e6dab80a3f2a99a18d217c124d88c113e725a WatchSource:0}: Error finding container 936f664e788f1e699697dceb9e4e6dab80a3f2a99a18d217c124d88c113e725a: Status 404 returned error can't find the container with id 936f664e788f1e699697dceb9e4e6dab80a3f2a99a18d217c124d88c113e725a
Sep 10 14:59:16.519110 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:16.519067    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm" event={"ID":"775ee203-8230-413a-8455-33757443992b","Type":"ContainerStarted","Data":"a3f8318442413c1b316256643fd5c573ca86ec59d1d07a8f8c47aad3b696a793"}
Sep 10 14:59:16.519612 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:16.519132    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm" event={"ID":"775ee203-8230-413a-8455-33757443992b","Type":"ContainerStarted","Data":"936f664e788f1e699697dceb9e4e6dab80a3f2a99a18d217c124d88c113e725a"}
Sep 10 14:59:16.537424 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:16.537374    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm" podStartSLOduration=1.5373588790000001 podStartE2EDuration="1.537358879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:16.534620856 +0000 UTC m=+1001.021308747" watchObservedRunningTime="2026-09-10 14:59:16.537358879 +0000 UTC m=+1001.024046770"
Sep 10 14:59:17.527346 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:17.527310    2558 generic.go:356] "Generic (PLEG): container finished" podID="775ee203-8230-413a-8455-33757443992b" containerID="a3f8318442413c1b316256643fd5c573ca86ec59d1d07a8f8c47aad3b696a793" exitCode=0
Sep 10 14:59:17.527745 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:17.527402    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm" event={"ID":"775ee203-8230-413a-8455-33757443992b","Type":"ContainerDied","Data":"a3f8318442413c1b316256643fd5c573ca86ec59d1d07a8f8c47aad3b696a793"}
Sep 10 14:59:18.664437 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:18.664413    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"
Sep 10 14:59:18.761655 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:18.761619    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ee203-8230-413a-8455-33757443992b-maas-api-service-ca\") pod \"775ee203-8230-413a-8455-33757443992b\" (UID: \"775ee203-8230-413a-8455-33757443992b\") "
Sep 10 14:59:18.762028 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:18.762004    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/775ee203-8230-413a-8455-33757443992b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "775ee203-8230-413a-8455-33757443992b" (UID: "775ee203-8230-413a-8455-33757443992b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:59:18.863328 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:18.863244    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/775ee203-8230-413a-8455-33757443992b-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:19.542334 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:19.542279    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm" event={"ID":"775ee203-8230-413a-8455-33757443992b","Type":"ContainerDied","Data":"936f664e788f1e699697dceb9e4e6dab80a3f2a99a18d217c124d88c113e725a"}
Sep 10 14:59:19.542496 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:19.542351    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="936f664e788f1e699697dceb9e4e6dab80a3f2a99a18d217c124d88c113e725a"
Sep 10 14:59:19.542496 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:19.542304    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"
Sep 10 14:59:25.877817 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:25.877781    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:59:26.418427 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:26.418390    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"]
Sep 10 14:59:26.423265 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:26.423240    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ff661a-xlzmm"]
Sep 10 14:59:28.019750 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:28.019713    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="775ee203-8230-413a-8455-33757443992b" path="/var/lib/kubelet/pods/775ee203-8230-413a-8455-33757443992b/volumes"
Sep 10 14:59:33.955740 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:33.955707    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"]
Sep 10 14:59:33.956208 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:33.956114    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775ee203-8230-413a-8455-33757443992b" containerName="revoke-keys"
Sep 10 14:59:33.956208 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:33.956129    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="775ee203-8230-413a-8455-33757443992b" containerName="revoke-keys"
Sep 10 14:59:33.956208 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:33.956198    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="775ee203-8230-413a-8455-33757443992b" containerName="revoke-keys"
Sep 10 14:59:33.959070 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:33.959054    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:33.961703 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:33.961666    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-16ad45-openshift-default-dockercfg-9gxz2\""
Sep 10 14:59:33.973335 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:33.973309    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"]
Sep 10 14:59:34.111125 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111088    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-envoy\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111145    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-token\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111200    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7fl8\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-kube-api-access-m7fl8\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd8417-22b4-4689-969a-23fb724f9ee3-istiod-ca-cert\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111292    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-socket\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111309    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-data\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111333 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-podinfo\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-certs\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.111559 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.111418    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-credential-socket\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212286 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-envoy\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212458 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212311    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-token\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212458 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212333    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7fl8\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-kube-api-access-m7fl8\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212458 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212358    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd8417-22b4-4689-969a-23fb724f9ee3-istiod-ca-cert\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212458 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212392    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-socket\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212458 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212425    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-data\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212458 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212450    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-podinfo\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212761 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212487    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-certs\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212761 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212518    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-credential-socket\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.212870 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212801    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-socket\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.213023 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.212998    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-data\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.213090 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.213035    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-credential-socket\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.213148 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.213131    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd8417-22b4-4689-969a-23fb724f9ee3-istiod-ca-cert\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.213207 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.213191    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-certs\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.214799 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.214778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-envoy\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.214997 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.214980    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-podinfo\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.223235 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.223198    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-token\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.223714 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.223695    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7fl8\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-kube-api-access-m7fl8\") pod \"e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") " pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.270821 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.270790    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:34.550405 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.550375    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"]
Sep 10 14:59:34.551995 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:59:34.551969    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bd8417_22b4_4689_969a_23fb724f9ee3.slice/crio-04559d51734a2be1408da620d16c2d43faef2b73f608d8e615295ffd4ccc0007 WatchSource:0}: Error finding container 04559d51734a2be1408da620d16c2d43faef2b73f608d8e615295ffd4ccc0007: Status 404 returned error can't find the container with id 04559d51734a2be1408da620d16c2d43faef2b73f608d8e615295ffd4ccc0007
Sep 10 14:59:34.554697 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.554665    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:34.554770 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.554734    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:34.554770 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.554764    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:34.630905 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.630867    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" event={"ID":"33bd8417-22b4-4689-969a-23fb724f9ee3","Type":"ContainerStarted","Data":"5412e678a174ae076f200d7b477ddc5d5283368f21432ab48f7d3446a6ac11da"}
Sep 10 14:59:34.631060 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.630919    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" event={"ID":"33bd8417-22b4-4689-969a-23fb724f9ee3","Type":"ContainerStarted","Data":"04559d51734a2be1408da620d16c2d43faef2b73f608d8e615295ffd4ccc0007"}
Sep 10 14:59:34.649692 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:34.649633    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podStartSLOduration=1.649611275 podStartE2EDuration="1.649611275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:34.649091483 +0000 UTC m=+1019.135779377" watchObservedRunningTime="2026-09-10 14:59:34.649611275 +0000 UTC m=+1019.136299169"
Sep 10 14:59:35.271593 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:35.271560    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:35.276498 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:35.276472    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:35.637420 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:35.637321    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:35.638453 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:35.638430    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:37.469278 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:37.469245    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:59:38.794522 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:38.794490    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7656-08a5a3"]
Sep 10 14:59:38.799416 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:38.799395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:38.802037 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:38.802013    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mmqf2\""
Sep 10 14:59:38.803958 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:38.803931    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-08a5a3"]
Sep 10 14:59:38.848946 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:38.848901    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvq4d\" (UniqueName: \"kubernetes.io/projected/fd12df85-864b-4e59-9d4e-554e9d2647ee-kube-api-access-lvq4d\") pod \"test-curl-7656-08a5a3\" (UID: \"fd12df85-864b-4e59-9d4e-554e9d2647ee\") " pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:38.949563 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:38.949532    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq4d\" (UniqueName: \"kubernetes.io/projected/fd12df85-864b-4e59-9d4e-554e9d2647ee-kube-api-access-lvq4d\") pod \"test-curl-7656-08a5a3\" (UID: \"fd12df85-864b-4e59-9d4e-554e9d2647ee\") " pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:38.957799 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:38.957767    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvq4d\" (UniqueName: \"kubernetes.io/projected/fd12df85-864b-4e59-9d4e-554e9d2647ee-kube-api-access-lvq4d\") pod \"test-curl-7656-08a5a3\" (UID: \"fd12df85-864b-4e59-9d4e-554e9d2647ee\") " pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:39.110067 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:39.109956    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:39.381660 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:39.381638    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-08a5a3"]
Sep 10 14:59:39.383890 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:59:39.383862    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd12df85_864b_4e59_9d4e_554e9d2647ee.slice/crio-71749092a9838a5bafec0fe01ee84bdefcf049da52c8021840166105cfc35003 WatchSource:0}: Error finding container 71749092a9838a5bafec0fe01ee84bdefcf049da52c8021840166105cfc35003: Status 404 returned error can't find the container with id 71749092a9838a5bafec0fe01ee84bdefcf049da52c8021840166105cfc35003
Sep 10 14:59:39.658705 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:39.658615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-08a5a3" event={"ID":"fd12df85-864b-4e59-9d4e-554e9d2647ee","Type":"ContainerStarted","Data":"71749092a9838a5bafec0fe01ee84bdefcf049da52c8021840166105cfc35003"}
Sep 10 14:59:42.679965 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:42.679932    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-08a5a3" event={"ID":"fd12df85-864b-4e59-9d4e-554e9d2647ee","Type":"ContainerStarted","Data":"2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef"}
Sep 10 14:59:42.695834 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:42.695779    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7656-08a5a3" podStartSLOduration=1.511025007 podStartE2EDuration="4.695762832s" totalImagesPullingTime="3.184737825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:42.694447947 +0000 UTC m=+1027.181135838" watchObservedRunningTime="2026-09-10 14:59:42.695762832 +0000 UTC m=+1027.182450727"
Sep 10 14:59:43.302460 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.302428    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7656-08a5a3"]
Sep 10 14:59:43.306699 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.306671    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7656-08a5a3"]
Sep 10 14:59:43.435650 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.435614    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7656-46a9ce"]
Sep 10 14:59:43.436012 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.435999    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd12df85-864b-4e59-9d4e-554e9d2647ee" containerName="test-curl-7656-08a5a3"
Sep 10 14:59:43.436056 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.436014    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd12df85-864b-4e59-9d4e-554e9d2647ee" containerName="test-curl-7656-08a5a3"
Sep 10 14:59:43.436095 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.436083    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd12df85-864b-4e59-9d4e-554e9d2647ee" containerName="test-curl-7656-08a5a3"
Sep 10 14:59:43.439383 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.439365    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:43.446367 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.446326    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-46a9ce"]
Sep 10 14:59:43.492600 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.492567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xjs7\" (UniqueName: \"kubernetes.io/projected/c7827ede-3e31-45e2-b6c2-5e2c1d836f0f-kube-api-access-6xjs7\") pod \"test-curl-7656-46a9ce\" (UID: \"c7827ede-3e31-45e2-b6c2-5e2c1d836f0f\") " pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:43.593679 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.593597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xjs7\" (UniqueName: \"kubernetes.io/projected/c7827ede-3e31-45e2-b6c2-5e2c1d836f0f-kube-api-access-6xjs7\") pod \"test-curl-7656-46a9ce\" (UID: \"c7827ede-3e31-45e2-b6c2-5e2c1d836f0f\") " pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:43.602134 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.602103    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xjs7\" (UniqueName: \"kubernetes.io/projected/c7827ede-3e31-45e2-b6c2-5e2c1d836f0f-kube-api-access-6xjs7\") pod \"test-curl-7656-46a9ce\" (UID: \"c7827ede-3e31-45e2-b6c2-5e2c1d836f0f\") " pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:43.686326 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:43.686293    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7656-08a5a3" container="test-curl-7656-08a5a3"
Sep 10 14:59:43.751111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:43.751079    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:44.024194 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:44.024169    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-46a9ce"]
Sep 10 14:59:44.025880 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:59:44.025853    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7827ede_3e31_45e2_b6c2_5e2c1d836f0f.slice/crio-2299a08a5086e72c04a72e9ffd2ea37cc7881f697042a337d8d52d4c0f3b3508 WatchSource:0}: Error finding container 2299a08a5086e72c04a72e9ffd2ea37cc7881f697042a337d8d52d4c0f3b3508: Status 404 returned error can't find the container with id 2299a08a5086e72c04a72e9ffd2ea37cc7881f697042a337d8d52d4c0f3b3508
Sep 10 14:59:44.700129 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:44.700094    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-46a9ce" event={"ID":"c7827ede-3e31-45e2-b6c2-5e2c1d836f0f","Type":"ContainerStarted","Data":"856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1"}
Sep 10 14:59:44.700634 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:44.700137    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-46a9ce" event={"ID":"c7827ede-3e31-45e2-b6c2-5e2c1d836f0f","Type":"ContainerStarted","Data":"2299a08a5086e72c04a72e9ffd2ea37cc7881f697042a337d8d52d4c0f3b3508"}
Sep 10 14:59:44.700634 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:44.700279    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7656-08a5a3" podUID="fd12df85-864b-4e59-9d4e-554e9d2647ee" containerName="test-curl-7656-08a5a3" containerID="cri-o://2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef" gracePeriod=2
Sep 10 14:59:44.716963 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:44.716919    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7656-46a9ce" podStartSLOduration=1.7169047960000001 podStartE2EDuration="1.716904796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:44.714348262 +0000 UTC m=+1029.201036152" watchObservedRunningTime="2026-09-10 14:59:44.716904796 +0000 UTC m=+1029.203592687"
Sep 10 14:59:45.318752 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.318716    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7656-46a9ce"]
Sep 10 14:59:45.322839 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.322815    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7656-46a9ce"]
Sep 10 14:59:45.443807 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.443772    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7656-62bdd0"]
Sep 10 14:59:45.444299 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.444280    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" containerName="test-curl-7656-46a9ce"
Sep 10 14:59:45.444403 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.444301    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" containerName="test-curl-7656-46a9ce"
Sep 10 14:59:45.444462 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.444403    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" containerName="test-curl-7656-46a9ce"
Sep 10 14:59:45.447648 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.447629    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:45.453255 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.453229    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-62bdd0"]
Sep 10 14:59:45.510869 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.510833    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzlf4\" (UniqueName: \"kubernetes.io/projected/8549ff34-b2e8-493a-b4b5-e8c5c89d9225-kube-api-access-zzlf4\") pod \"test-curl-7656-62bdd0\" (UID: \"8549ff34-b2e8-493a-b4b5-e8c5c89d9225\") " pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:45.611987 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.611946    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzlf4\" (UniqueName: \"kubernetes.io/projected/8549ff34-b2e8-493a-b4b5-e8c5c89d9225-kube-api-access-zzlf4\") pod \"test-curl-7656-62bdd0\" (UID: \"8549ff34-b2e8-493a-b4b5-e8c5c89d9225\") " pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:45.619847 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.619821    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzlf4\" (UniqueName: \"kubernetes.io/projected/8549ff34-b2e8-493a-b4b5-e8c5c89d9225-kube-api-access-zzlf4\") pod \"test-curl-7656-62bdd0\" (UID: \"8549ff34-b2e8-493a-b4b5-e8c5c89d9225\") " pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:45.709871 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:45.709768    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7656-46a9ce" container="test-curl-7656-46a9ce"
Sep 10 14:59:45.758365 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:45.758314    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:46.057744 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:46.057713    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-62bdd0"]
Sep 10 14:59:46.059010 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:59:46.058981    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8549ff34_b2e8_493a_b4b5_e8c5c89d9225.slice/crio-00f8f8aa4d872de75d5059370c2f6448e10b79e4ee3cb2962605fad30d5ea1d6 WatchSource:0}: Error finding container 00f8f8aa4d872de75d5059370c2f6448e10b79e4ee3cb2962605fad30d5ea1d6: Status 404 returned error can't find the container with id 00f8f8aa4d872de75d5059370c2f6448e10b79e4ee3cb2962605fad30d5ea1d6
Sep 10 14:59:46.717202 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:46.717173    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-62bdd0" event={"ID":"8549ff34-b2e8-493a-b4b5-e8c5c89d9225","Type":"ContainerStarted","Data":"30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795"}
Sep 10 14:59:46.717579 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:46.717262    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-62bdd0" event={"ID":"8549ff34-b2e8-493a-b4b5-e8c5c89d9225","Type":"ContainerStarted","Data":"00f8f8aa4d872de75d5059370c2f6448e10b79e4ee3cb2962605fad30d5ea1d6"}
Sep 10 14:59:46.717579 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:46.717240    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7656-46a9ce" podUID="c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" containerName="test-curl-7656-46a9ce" containerID="cri-o://856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1" gracePeriod=2
Sep 10 14:59:46.735947 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:46.735898    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7656-62bdd0" podStartSLOduration=1.735881961 podStartE2EDuration="1.735881961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:46.733952472 +0000 UTC m=+1031.220640380" watchObservedRunningTime="2026-09-10 14:59:46.735881961 +0000 UTC m=+1031.222569906"
Sep 10 14:59:46.934821 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:46.934797    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:47.025558 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.025464    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvq4d\" (UniqueName: \"kubernetes.io/projected/fd12df85-864b-4e59-9d4e-554e9d2647ee-kube-api-access-lvq4d\") pod \"fd12df85-864b-4e59-9d4e-554e9d2647ee\" (UID: \"fd12df85-864b-4e59-9d4e-554e9d2647ee\") "
Sep 10 14:59:47.027812 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.027787    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd12df85-864b-4e59-9d4e-554e9d2647ee-kube-api-access-lvq4d" (OuterVolumeSpecName: "kube-api-access-lvq4d") pod "fd12df85-864b-4e59-9d4e-554e9d2647ee" (UID: "fd12df85-864b-4e59-9d4e-554e9d2647ee"). InnerVolumeSpecName "kube-api-access-lvq4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:59:47.126750 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.126717    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvq4d\" (UniqueName: \"kubernetes.io/projected/fd12df85-864b-4e59-9d4e-554e9d2647ee-kube-api-access-lvq4d\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:47.378608 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.378508    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7656-62bdd0"]
Sep 10 14:59:47.383509 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.383466    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7656-62bdd0"]
Sep 10 14:59:47.505763 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.505726    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7656-82411c"]
Sep 10 14:59:47.506150 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.506135    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8549ff34-b2e8-493a-b4b5-e8c5c89d9225" containerName="test-curl-7656-62bdd0"
Sep 10 14:59:47.506195 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.506152    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8549ff34-b2e8-493a-b4b5-e8c5c89d9225" containerName="test-curl-7656-62bdd0"
Sep 10 14:59:47.506257 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.506249    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8549ff34-b2e8-493a-b4b5-e8c5c89d9225" containerName="test-curl-7656-62bdd0"
Sep 10 14:59:47.509181 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.509164    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:47.515545 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.515524    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-82411c"]
Sep 10 14:59:47.530507 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.530474    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/61f5356b-18cc-4145-83a1-c5ccfaecfda1-kube-api-access-h4724\") pod \"test-curl-7656-82411c\" (UID: \"61f5356b-18cc-4145-83a1-c5ccfaecfda1\") " pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:47.631449 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.631357    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/61f5356b-18cc-4145-83a1-c5ccfaecfda1-kube-api-access-h4724\") pod \"test-curl-7656-82411c\" (UID: \"61f5356b-18cc-4145-83a1-c5ccfaecfda1\") " pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:47.640701 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.640670    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/61f5356b-18cc-4145-83a1-c5ccfaecfda1-kube-api-access-h4724\") pod \"test-curl-7656-82411c\" (UID: \"61f5356b-18cc-4145-83a1-c5ccfaecfda1\") " pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:47.683391 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.683344    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:59:47.724443 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.724414    2558 generic.go:356] "Generic (PLEG): container finished" podID="fd12df85-864b-4e59-9d4e-554e9d2647ee" containerID="2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef" exitCode=137
Sep 10 14:59:47.724895 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.724453    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:47.724895 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.724516    2558 scope.go:117] "RemoveContainer" containerID="2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef"
Sep 10 14:59:47.724895 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:47.724690    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7656-62bdd0" container="test-curl-7656-62bdd0"
Sep 10 14:59:47.738240 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.734412    2558 scope.go:117] "RemoveContainer" containerID="2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef"
Sep 10 14:59:47.738240 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:47.734765    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef\": container with ID starting with 2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef not found: ID does not exist" containerID="2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef"
Sep 10 14:59:47.738240 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.734809    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef"} err="failed to get container status \"2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef\": rpc error: code = NotFound desc = could not find container \"2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef\": container with ID starting with 2350950040a6f0b2c1e19da90d6bf4b54a00160f04d924af62619eecad9363ef not found: ID does not exist"
Sep 10 14:59:47.740180 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:47.740151    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7656-08a5a3\" is forbidden: User \"system:node:ip-10-0-142-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-253.ec2.internal' and this object" podUID="fd12df85-864b-4e59-9d4e-554e9d2647ee" pod="opendatahub/test-curl-7656-08a5a3"
Sep 10 14:59:47.820336 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:47.820292    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:48.019076 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:48.019038    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd12df85-864b-4e59-9d4e-554e9d2647ee" path="/var/lib/kubelet/pods/fd12df85-864b-4e59-9d4e-554e9d2647ee/volumes"
Sep 10 14:59:48.103895 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:48.103858    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7656-82411c"]
Sep 10 14:59:48.109229 ip-10-0-142-253 kubenswrapper[2558]: W0910 14:59:48.109174    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61f5356b_18cc_4145_83a1_c5ccfaecfda1.slice/crio-6195c75be3ff8936ce5e63d15490e08aed2708865a607967640c3323db43c606 WatchSource:0}: Error finding container 6195c75be3ff8936ce5e63d15490e08aed2708865a607967640c3323db43c606: Status 404 returned error can't find the container with id 6195c75be3ff8936ce5e63d15490e08aed2708865a607967640c3323db43c606
Sep 10 14:59:48.733373 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:48.733342    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-82411c" event={"ID":"61f5356b-18cc-4145-83a1-c5ccfaecfda1","Type":"ContainerStarted","Data":"bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c"}
Sep 10 14:59:48.733716 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:48.733385    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7656-82411c" event={"ID":"61f5356b-18cc-4145-83a1-c5ccfaecfda1","Type":"ContainerStarted","Data":"6195c75be3ff8936ce5e63d15490e08aed2708865a607967640c3323db43c606"}
Sep 10 14:59:48.733716 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:48.733484    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7656-62bdd0" podUID="8549ff34-b2e8-493a-b4b5-e8c5c89d9225" containerName="test-curl-7656-62bdd0" containerID="cri-o://30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795" gracePeriod=2
Sep 10 14:59:48.931056 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:48.931032    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:49.045528 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.045440    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xjs7\" (UniqueName: \"kubernetes.io/projected/c7827ede-3e31-45e2-b6c2-5e2c1d836f0f-kube-api-access-6xjs7\") pod \"c7827ede-3e31-45e2-b6c2-5e2c1d836f0f\" (UID: \"c7827ede-3e31-45e2-b6c2-5e2c1d836f0f\") "
Sep 10 14:59:49.047618 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.047585    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7827ede-3e31-45e2-b6c2-5e2c1d836f0f-kube-api-access-6xjs7" (OuterVolumeSpecName: "kube-api-access-6xjs7") pod "c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" (UID: "c7827ede-3e31-45e2-b6c2-5e2c1d836f0f"). InnerVolumeSpecName "kube-api-access-6xjs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:59:49.146857 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.146822    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xjs7\" (UniqueName: \"kubernetes.io/projected/c7827ede-3e31-45e2-b6c2-5e2c1d836f0f-kube-api-access-6xjs7\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:49.429595 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.429490    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7656-82411c" podStartSLOduration=2.42946902 podStartE2EDuration="2.42946902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:48.754527634 +0000 UTC m=+1033.241215525" watchObservedRunningTime="2026-09-10 14:59:49.42946902 +0000 UTC m=+1033.916156911"
Sep 10 14:59:49.431184 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.431165    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7656-82411c"]
Sep 10 14:59:49.435041 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.435013    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7656-82411c"]
Sep 10 14:59:49.740900 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.740866    2558 generic.go:356] "Generic (PLEG): container finished" podID="c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" containerID="856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1" exitCode=137
Sep 10 14:59:49.741396 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.740910    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:49.741396 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.740968    2558 scope.go:117] "RemoveContainer" containerID="856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1"
Sep 10 14:59:49.741396 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:49.741268    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7656-82411c" container="test-curl-7656-82411c"
Sep 10 14:59:49.750888 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.750871    2558 scope.go:117] "RemoveContainer" containerID="856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1"
Sep 10 14:59:49.751180 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:49.751163    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1\": container with ID starting with 856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1 not found: ID does not exist" containerID="856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1"
Sep 10 14:59:49.751247 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:49.751190    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1"} err="failed to get container status \"856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1\": rpc error: code = NotFound desc = could not find container \"856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1\": container with ID starting with 856108559f6342793c74185a69d90a8ca1b02fbcde475404588d5e72c5ca96e1 not found: ID does not exist"
Sep 10 14:59:49.751651 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:49.751628    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7656-46a9ce\" is forbidden: User \"system:node:ip-10-0-142-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-253.ec2.internal' and this object" podUID="c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" pod="opendatahub/test-curl-7656-46a9ce"
Sep 10 14:59:50.019374 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:50.019284    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7827ede-3e31-45e2-b6c2-5e2c1d836f0f" path="/var/lib/kubelet/pods/c7827ede-3e31-45e2-b6c2-5e2c1d836f0f/volumes"
Sep 10 14:59:50.751114 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:50.751081    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7656-82411c" podUID="61f5356b-18cc-4145-83a1-c5ccfaecfda1" containerName="test-curl-7656-82411c" containerID="cri-o://bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c" gracePeriod=2
Sep 10 14:59:50.873679 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:50.873650    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:50.963467 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:50.963432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzlf4\" (UniqueName: \"kubernetes.io/projected/8549ff34-b2e8-493a-b4b5-e8c5c89d9225-kube-api-access-zzlf4\") pod \"8549ff34-b2e8-493a-b4b5-e8c5c89d9225\" (UID: \"8549ff34-b2e8-493a-b4b5-e8c5c89d9225\") "
Sep 10 14:59:50.965609 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:50.965583    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8549ff34-b2e8-493a-b4b5-e8c5c89d9225-kube-api-access-zzlf4" (OuterVolumeSpecName: "kube-api-access-zzlf4") pod "8549ff34-b2e8-493a-b4b5-e8c5c89d9225" (UID: "8549ff34-b2e8-493a-b4b5-e8c5c89d9225"). InnerVolumeSpecName "kube-api-access-zzlf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:59:51.064962 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.064925    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzlf4\" (UniqueName: \"kubernetes.io/projected/8549ff34-b2e8-493a-b4b5-e8c5c89d9225-kube-api-access-zzlf4\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:51.678411 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.678369    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 14:59:51.678568 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.678433    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 14:59:51.758866 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.758834    2558 generic.go:356] "Generic (PLEG): container finished" podID="8549ff34-b2e8-493a-b4b5-e8c5c89d9225" containerID="30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795" exitCode=137
Sep 10 14:59:51.759341 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.758873    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:51.759341 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.758958    2558 scope.go:117] "RemoveContainer" containerID="30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795"
Sep 10 14:59:51.770167 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.770143    2558 scope.go:117] "RemoveContainer" containerID="30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795"
Sep 10 14:59:51.770566 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:51.770542    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795\": container with ID starting with 30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795 not found: ID does not exist" containerID="30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795"
Sep 10 14:59:51.770640 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:51.770580    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795"} err="failed to get container status \"30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795\": rpc error: code = NotFound desc = could not find container \"30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795\": container with ID starting with 30de4adff712d6b8be00ffbc2c764a70a4d6907184d5e94dc748d2d04f071795 not found: ID does not exist"
Sep 10 14:59:51.771000 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:51.770970    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7656-62bdd0\" is forbidden: User \"system:node:ip-10-0-142-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-253.ec2.internal' and this object" podUID="8549ff34-b2e8-493a-b4b5-e8c5c89d9225" pod="opendatahub/test-curl-7656-62bdd0"
Sep 10 14:59:52.018765 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:52.018731    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8549ff34-b2e8-493a-b4b5-e8c5c89d9225" path="/var/lib/kubelet/pods/8549ff34-b2e8-493a-b4b5-e8c5c89d9225/volumes"
Sep 10 14:59:52.912051 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:52.912025    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:52.981592 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:52.981557    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/61f5356b-18cc-4145-83a1-c5ccfaecfda1-kube-api-access-h4724\") pod \"61f5356b-18cc-4145-83a1-c5ccfaecfda1\" (UID: \"61f5356b-18cc-4145-83a1-c5ccfaecfda1\") "
Sep 10 14:59:52.983750 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:52.983723    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61f5356b-18cc-4145-83a1-c5ccfaecfda1-kube-api-access-h4724" (OuterVolumeSpecName: "kube-api-access-h4724") pod "61f5356b-18cc-4145-83a1-c5ccfaecfda1" (UID: "61f5356b-18cc-4145-83a1-c5ccfaecfda1"). InnerVolumeSpecName "kube-api-access-h4724". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:59:53.082717 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.082639    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4724\" (UniqueName: \"kubernetes.io/projected/61f5356b-18cc-4145-83a1-c5ccfaecfda1-kube-api-access-h4724\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 14:59:53.784165 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.784133    2558 generic.go:356] "Generic (PLEG): container finished" podID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerID="5412e678a174ae076f200d7b477ddc5d5283368f21432ab48f7d3446a6ac11da" exitCode=0
Sep 10 14:59:53.784432 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.784246    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" event={"ID":"33bd8417-22b4-4689-969a-23fb724f9ee3","Type":"ContainerDied","Data":"5412e678a174ae076f200d7b477ddc5d5283368f21432ab48f7d3446a6ac11da"}
Sep 10 14:59:53.784735 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.784712    2558 scope.go:117] "RemoveContainer" containerID="5412e678a174ae076f200d7b477ddc5d5283368f21432ab48f7d3446a6ac11da"
Sep 10 14:59:53.785806 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.785767    2558 generic.go:356] "Generic (PLEG): container finished" podID="61f5356b-18cc-4145-83a1-c5ccfaecfda1" containerID="bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c" exitCode=137
Sep 10 14:59:53.785909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.785812    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:53.785909 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.785842    2558 scope.go:117] "RemoveContainer" containerID="bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c"
Sep 10 14:59:53.786003 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.785911    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:53.786003 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.785982    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:53.786082 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.786036    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 14:59:53.802570 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.802546    2558 scope.go:117] "RemoveContainer" containerID="bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c"
Sep 10 14:59:53.802910 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:53.802890    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c\": container with ID starting with bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c not found: ID does not exist" containerID="bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c"
Sep 10 14:59:53.802958 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:53.802920    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c"} err="failed to get container status \"bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c\": rpc error: code = NotFound desc = could not find container \"bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c\": container with ID starting with bc02ef84453843cfbf13b598c55d077871f8fe1647f2ee21c954609029b7035c not found: ID does not exist"
Sep 10 14:59:53.833788 ip-10-0-142-253 kubenswrapper[2558]: E0910 14:59:53.833729    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7656-82411c\" is forbidden: User \"system:node:ip-10-0-142-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-253.ec2.internal' and this object" podUID="61f5356b-18cc-4145-83a1-c5ccfaecfda1" pod="opendatahub/test-curl-7656-82411c"
Sep 10 14:59:54.020111 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:54.020077    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61f5356b-18cc-4145-83a1-c5ccfaecfda1" path="/var/lib/kubelet/pods/61f5356b-18cc-4145-83a1-c5ccfaecfda1/volumes"
Sep 10 14:59:54.799911 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:54.799869    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" event={"ID":"33bd8417-22b4-4689-969a-23fb724f9ee3","Type":"ContainerStarted","Data":"e5691eb527d941f8446beb59be84aabfc5831234d2cf28553e752fb98711cfe4"}
Sep 10 14:59:55.271011 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:55.270976    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 14:59:55.271428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:55.271155    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 10 14:59:55.271428 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:55.271197    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 10 14:59:56.271263 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:56.271205    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 10 14:59:56.271729 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:56.271283    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 10 14:59:57.271828 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:57.271792    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 10 14:59:57.272320 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:57.271855    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 10 14:59:58.271495 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:58.271451    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 10 14:59:58.271701 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:58.271517    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 10 14:59:59.098599 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:59.098536    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 14:59:59.271422 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:59.271384    2558 patch_prober.go:28] interesting pod/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 10 14:59:59.271588 ip-10-0-142-253 kubenswrapper[2558]: I0910 14:59:59.271454    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 10 15:00:00.147361 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.147327    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"]
Sep 10 15:00:00.147770 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.147756    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61f5356b-18cc-4145-83a1-c5ccfaecfda1" containerName="test-curl-7656-82411c"
Sep 10 15:00:00.147818 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.147772    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="61f5356b-18cc-4145-83a1-c5ccfaecfda1" containerName="test-curl-7656-82411c"
Sep 10 15:00:00.147857 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.147847    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="61f5356b-18cc-4145-83a1-c5ccfaecfda1" containerName="test-curl-7656-82411c"
Sep 10 15:00:00.154543 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.154517    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"
Sep 10 15:00:00.159140 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.159114    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:00:00.164187 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.164166    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"]
Sep 10 15:00:00.172936 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.170203    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"]
Sep 10 15:00:00.172936 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.170333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"
Sep 10 15:00:00.173683 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.173663    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"
Sep 10 15:00:00.175463 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.175444    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"]
Sep 10 15:00:00.175597 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.175574    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"
Sep 10 15:00:00.188211 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.188186    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"
Sep 10 15:00:00.200300 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.200070    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"]
Sep 10 15:00:00.200300 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.200135    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"]
Sep 10 15:00:00.237053 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.237026    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"
Sep 10 15:00:00.450042 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.450011    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 15:00:00.450446 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.450397    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 15:00:00.451476 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.451455    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 15:00:00.480266 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.480206    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"]
Sep 10 15:00:00.482697 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:00:00.482674    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaea05a25_bdad_4802_8908_8a692a320774.slice/crio-9b08942b6d2f5ee74e41149b25ae4309b1da7d13b3b6b09869b071d5a4bd2089 WatchSource:0}: Error finding container 9b08942b6d2f5ee74e41149b25ae4309b1da7d13b3b6b09869b071d5a4bd2089: Status 404 returned error can't find the container with id 9b08942b6d2f5ee74e41149b25ae4309b1da7d13b3b6b09869b071d5a4bd2089
Sep 10 15:00:00.495309 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.495275    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"]
Sep 10 15:00:00.497417 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:00:00.497391    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02164c8d_05a4_4714_bfbe_8f0e8c7d3280.slice/crio-e7312d85461038c42f9304ad34f85cdf57b1a041e8d5652f7c931c3fd333bc7c WatchSource:0}: Error finding container e7312d85461038c42f9304ad34f85cdf57b1a041e8d5652f7c931c3fd333bc7c: Status 404 returned error can't find the container with id e7312d85461038c42f9304ad34f85cdf57b1a041e8d5652f7c931c3fd333bc7c
Sep 10 15:00:00.562236 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.562186    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"]
Sep 10 15:00:00.564041 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:00:00.564017    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd61dbf_093e_4824_bc80_46129214542f.slice/crio-4f5ae8685f509f882a315bca08b8d02c0bf66dc0dc07872f2064e23f4b05f75e WatchSource:0}: Error finding container 4f5ae8685f509f882a315bca08b8d02c0bf66dc0dc07872f2064e23f4b05f75e: Status 404 returned error can't find the container with id 4f5ae8685f509f882a315bca08b8d02c0bf66dc0dc07872f2064e23f4b05f75e
Sep 10 15:00:00.837774 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.837733    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6" event={"ID":"8fd61dbf-093e-4824-bc80-46129214542f","Type":"ContainerStarted","Data":"4f5ae8685f509f882a315bca08b8d02c0bf66dc0dc07872f2064e23f4b05f75e"}
Sep 10 15:00:00.839098 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.839066    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" event={"ID":"02164c8d-05a4-4714-bfbe-8f0e8c7d3280","Type":"ContainerStarted","Data":"e7312d85461038c42f9304ad34f85cdf57b1a041e8d5652f7c931c3fd333bc7c"}
Sep 10 15:00:00.840715 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.840685    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x" event={"ID":"aea05a25-bdad-4802-8908-8a692a320774","Type":"ContainerStarted","Data":"496e032da7a0f0b729f614174a6bdf548e8f8337f0a96e1ada50a96d36be4a8f"}
Sep 10 15:00:00.840829 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.840735    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x" event={"ID":"aea05a25-bdad-4802-8908-8a692a320774","Type":"ContainerStarted","Data":"9b08942b6d2f5ee74e41149b25ae4309b1da7d13b3b6b09869b071d5a4bd2089"}
Sep 10 15:00:00.856810 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:00.856763    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x" podStartSLOduration=0.856744158 podStartE2EDuration="856.744158ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:00.856514222 +0000 UTC m=+1045.343202114" watchObservedRunningTime="2026-09-10 15:00:00.856744158 +0000 UTC m=+1045.343432053"
Sep 10 15:00:01.848457 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:01.848420    2558 generic.go:356] "Generic (PLEG): container finished" podID="8fd61dbf-093e-4824-bc80-46129214542f" containerID="2b5383035093ca2f474fa380002e89f3c9aaccc849b38c2e459ab787b06a7dc6" exitCode=0
Sep 10 15:00:01.848913 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:01.848492    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6" event={"ID":"8fd61dbf-093e-4824-bc80-46129214542f","Type":"ContainerDied","Data":"2b5383035093ca2f474fa380002e89f3c9aaccc849b38c2e459ab787b06a7dc6"}
Sep 10 15:00:01.850108 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:01.850079    2558 generic.go:356] "Generic (PLEG): container finished" podID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerID="1a8d5f117536b18e22ab5aae835c286454c0aeb58dd1e34d97fa33e5ccb8276b" exitCode=7
Sep 10 15:00:01.850244 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:01.850163    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" event={"ID":"02164c8d-05a4-4714-bfbe-8f0e8c7d3280","Type":"ContainerDied","Data":"1a8d5f117536b18e22ab5aae835c286454c0aeb58dd1e34d97fa33e5ccb8276b"}
Sep 10 15:00:01.850326 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:01.850313    2558 scope.go:117] "RemoveContainer" containerID="1a8d5f117536b18e22ab5aae835c286454c0aeb58dd1e34d97fa33e5ccb8276b"
Sep 10 15:00:01.851690 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:01.851668    2558 generic.go:356] "Generic (PLEG): container finished" podID="aea05a25-bdad-4802-8908-8a692a320774" containerID="496e032da7a0f0b729f614174a6bdf548e8f8337f0a96e1ada50a96d36be4a8f" exitCode=0
Sep 10 15:00:01.851776 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:01.851742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x" event={"ID":"aea05a25-bdad-4802-8908-8a692a320774","Type":"ContainerDied","Data":"496e032da7a0f0b729f614174a6bdf548e8f8337f0a96e1ada50a96d36be4a8f"}
Sep 10 15:00:02.255046 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.255011    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"]
Sep 10 15:00:02.258849 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.258832    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.262579 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.262557    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-3f81b7-openshift-default-dockercfg-p52sn\""
Sep 10 15:00:02.294823 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.294792    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"]
Sep 10 15:00:02.375997 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.375889    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-credential-socket\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.375997 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.375936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-envoy\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.376262 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.376050    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65ad5b87-68ce-4bae-8702-05416361c76f-istiod-ca-cert\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.376262 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.376186    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrqzg\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-kube-api-access-hrqzg\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.376358 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.376258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-socket\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.376358 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.376287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-certs\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.376358 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.376316    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65ad5b87-68ce-4bae-8702-05416361c76f-istio-podinfo\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.376485 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.376458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-data\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.376539 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.376504    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-istio-token\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477434 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477381    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-data\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477619 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-istio-token\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477619 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477505    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-credential-socket\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477619 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477530    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-envoy\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477619 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65ad5b87-68ce-4bae-8702-05416361c76f-istiod-ca-cert\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477868 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477635    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrqzg\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-kube-api-access-hrqzg\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477868 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477686    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-socket\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477868 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477714    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-certs\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.477868 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.477741    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65ad5b87-68ce-4bae-8702-05416361c76f-istio-podinfo\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.478080 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.478021    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-data\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.478146 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.478091    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-credential-socket\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.478202 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.478157    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-socket\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.478297 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.478239    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-certs\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.478485 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.478468    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65ad5b87-68ce-4bae-8702-05416361c76f-istiod-ca-cert\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.480120 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.480097    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-envoy\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.480641 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.480621    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65ad5b87-68ce-4bae-8702-05416361c76f-istio-podinfo\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.492483 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.492459    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-istio-token\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.494533 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.494515    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrqzg\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-kube-api-access-hrqzg\") pod \"e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") " pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.569740 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.569705    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:02.860340 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.860297    2558 generic.go:356] "Generic (PLEG): container finished" podID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerID="50df89e442d1921fea1c61eec11d071c1ba11e33a22eb567942a0ec66b41b8ae" exitCode=7
Sep 10 15:00:02.860815 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.860390    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" event={"ID":"02164c8d-05a4-4714-bfbe-8f0e8c7d3280","Type":"ContainerDied","Data":"50df89e442d1921fea1c61eec11d071c1ba11e33a22eb567942a0ec66b41b8ae"}
Sep 10 15:00:02.860815 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.860479    2558 scope.go:117] "RemoveContainer" containerID="1a8d5f117536b18e22ab5aae835c286454c0aeb58dd1e34d97fa33e5ccb8276b"
Sep 10 15:00:02.860815 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.860744    2558 scope.go:117] "RemoveContainer" containerID="50df89e442d1921fea1c61eec11d071c1ba11e33a22eb567942a0ec66b41b8ae"
Sep 10 15:00:02.861392 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:00:02.861023    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s_odh-ai-gateway-infra(02164c8d-05a4-4714-bfbe-8f0e8c7d3280)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280"
Sep 10 15:00:02.863313 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.863293    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"]
Sep 10 15:00:02.865253 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:00:02.865201    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65ad5b87_68ce_4bae_8702_05416361c76f.slice/crio-72cd867820d4611ed1b353903467ec24cafc2adf668734196338e9fec7abebc0 WatchSource:0}: Error finding container 72cd867820d4611ed1b353903467ec24cafc2adf668734196338e9fec7abebc0: Status 404 returned error can't find the container with id 72cd867820d4611ed1b353903467ec24cafc2adf668734196338e9fec7abebc0
Sep 10 15:00:02.867837 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.867799    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:00:02.867949 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.867891    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:00:02.867949 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:02.867935    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:00:03.008298 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.008274    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"
Sep 10 15:00:03.019457 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.019427    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"
Sep 10 15:00:03.869144 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.869055    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" event={"ID":"65ad5b87-68ce-4bae-8702-05416361c76f","Type":"ContainerStarted","Data":"6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3"}
Sep 10 15:00:03.869144 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.869126    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" event={"ID":"65ad5b87-68ce-4bae-8702-05416361c76f","Type":"ContainerStarted","Data":"72cd867820d4611ed1b353903467ec24cafc2adf668734196338e9fec7abebc0"}
Sep 10 15:00:03.870515 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.870471    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"
Sep 10 15:00:03.870639 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.870471    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6" event={"ID":"8fd61dbf-093e-4824-bc80-46129214542f","Type":"ContainerDied","Data":"4f5ae8685f509f882a315bca08b8d02c0bf66dc0dc07872f2064e23f4b05f75e"}
Sep 10 15:00:03.870639 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.870592    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f5ae8685f509f882a315bca08b8d02c0bf66dc0dc07872f2064e23f4b05f75e"
Sep 10 15:00:03.872266 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.872246    2558 scope.go:117] "RemoveContainer" containerID="50df89e442d1921fea1c61eec11d071c1ba11e33a22eb567942a0ec66b41b8ae"
Sep 10 15:00:03.872516 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:00:03.872494    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s_odh-ai-gateway-infra(02164c8d-05a4-4714-bfbe-8f0e8c7d3280)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280"
Sep 10 15:00:03.873291 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.873271    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"
Sep 10 15:00:03.873380 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.873285    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x" event={"ID":"aea05a25-bdad-4802-8908-8a692a320774","Type":"ContainerDied","Data":"9b08942b6d2f5ee74e41149b25ae4309b1da7d13b3b6b09869b071d5a4bd2089"}
Sep 10 15:00:03.873380 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.873327    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b08942b6d2f5ee74e41149b25ae4309b1da7d13b3b6b09869b071d5a4bd2089"
Sep 10 15:00:03.908340 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:03.908292    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podStartSLOduration=1.9082762739999999 podStartE2EDuration="1.908276274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:03.904144343 +0000 UTC m=+1048.390832234" watchObservedRunningTime="2026-09-10 15:00:03.908276274 +0000 UTC m=+1048.394964164"
Sep 10 15:00:04.570093 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:04.570057    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:04.574884 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:04.574861    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:04.879553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:04.879472    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:04.880434 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:04.880413    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:12.194572 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:12.194532    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:00:18.013609 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:18.013554    2558 scope.go:117] "RemoveContainer" containerID="50df89e442d1921fea1c61eec11d071c1ba11e33a22eb567942a0ec66b41b8ae"
Sep 10 15:00:18.972773 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:18.972736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" event={"ID":"02164c8d-05a4-4714-bfbe-8f0e8c7d3280","Type":"ContainerStarted","Data":"519682c8051ea97f2651177602acd18b706931519d75a78f42a1afe817491143"}
Sep 10 15:00:18.993654 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:18.993587    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" podStartSLOduration=18.993566329 podStartE2EDuration="18.993566329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:18.992916809 +0000 UTC m=+1063.479604700" watchObservedRunningTime="2026-09-10 15:00:18.993566329 +0000 UTC m=+1063.480254220"
Sep 10 15:00:19.987163 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:19.987122    2558 generic.go:356] "Generic (PLEG): container finished" podID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerID="519682c8051ea97f2651177602acd18b706931519d75a78f42a1afe817491143" exitCode=0
Sep 10 15:00:19.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:19.987168    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" event={"ID":"02164c8d-05a4-4714-bfbe-8f0e8c7d3280","Type":"ContainerDied","Data":"519682c8051ea97f2651177602acd18b706931519d75a78f42a1afe817491143"}
Sep 10 15:00:19.987584 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:19.987291    2558 scope.go:117] "RemoveContainer" containerID="50df89e442d1921fea1c61eec11d071c1ba11e33a22eb567942a0ec66b41b8ae"
Sep 10 15:00:20.007909 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:20.007869    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"]
Sep 10 15:00:20.922392 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:20.922348    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 15:00:20.922597 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:20.922423    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 15:00:21.135378 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:21.135352    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"
Sep 10 15:00:22.003178 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.003151    2558 generic.go:356] "Generic (PLEG): container finished" podID="65ad5b87-68ce-4bae-8702-05416361c76f" containerID="6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3" exitCode=0
Sep 10 15:00:22.003413 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.003250    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" event={"ID":"65ad5b87-68ce-4bae-8702-05416361c76f","Type":"ContainerDied","Data":"6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3"}
Sep 10 15:00:22.003742 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.003727    2558 scope.go:117] "RemoveContainer" containerID="6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3"
Sep 10 15:00:22.004730 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.004709    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"
Sep 10 15:00:22.004839 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.004723    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s" event={"ID":"02164c8d-05a4-4714-bfbe-8f0e8c7d3280","Type":"ContainerDied","Data":"e7312d85461038c42f9304ad34f85cdf57b1a041e8d5652f7c931c3fd333bc7c"}
Sep 10 15:00:22.004839 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.004788    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7312d85461038c42f9304ad34f85cdf57b1a041e8d5652f7c931c3fd333bc7c"
Sep 10 15:00:22.005010 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.004972    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:00:22.005081 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.005051    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:00:22.005138 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.005086    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:00:22.071339 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.071309    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"]
Sep 10 15:00:22.076367 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.076339    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6621560b-29817540-g586s"]
Sep 10 15:00:22.682145 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682108    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"]
Sep 10 15:00:22.682621 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682568    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682621 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682581    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682621 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682589    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fd61dbf-093e-4824-bc80-46129214542f" containerName="cleanup"
Sep 10 15:00:22.682621 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682596    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd61dbf-093e-4824-bc80-46129214542f" containerName="cleanup"
Sep 10 15:00:22.682621 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682615    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aea05a25-bdad-4802-8908-8a692a320774" containerName="cleanup"
Sep 10 15:00:22.682621 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682621    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="aea05a25-bdad-4802-8908-8a692a320774" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682630    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682635    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682644    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682649    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682708    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682715    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682722    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="aea05a25-bdad-4802-8908-8a692a320774" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682728    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" containerName="cleanup"
Sep 10 15:00:22.682932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.682734    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fd61dbf-093e-4824-bc80-46129214542f" containerName="cleanup"
Sep 10 15:00:22.684960 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.684936    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.689261 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.689237    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6621560b\"/\"e2e-lifecycle-model-6621560b-kserve-self-signed-certs\""
Sep 10 15:00:22.689990 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.689969    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6621560b\"/\"default-dockercfg-tkd75\""
Sep 10 15:00:22.689990 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.689996    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6621560b\"/\"kube-root-ca.crt\""
Sep 10 15:00:22.690194 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.689996    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6621560b\"/\"openshift-service-ca.crt\""
Sep 10 15:00:22.696844 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.696821    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"]
Sep 10 15:00:22.783151 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.783115    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-model-cache\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.783151 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.783147    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tls-certs\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.783424 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.783186    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-home\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.783424 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.783399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tmp-dir\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.783492 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.783464    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-dshm\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.783529 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.783493    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvqjt\" (UniqueName: \"kubernetes.io/projected/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-kube-api-access-tvqjt\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.884662 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.884627    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvqjt\" (UniqueName: \"kubernetes.io/projected/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-kube-api-access-tvqjt\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.884873 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.884682    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-model-cache\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.884873 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.884705    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tls-certs\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.884873 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.884740    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-home\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.884873 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.884782    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tmp-dir\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.884873 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.884833    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-dshm\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.885159 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.885075    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-model-cache\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.885159 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.885125    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-home\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.885288 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.885180    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tmp-dir\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.887276 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.887253    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-dshm\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.887499 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.887481    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tls-certs\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.894561 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.894537    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvqjt\" (UniqueName: \"kubernetes.io/projected/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-kube-api-access-tvqjt\") pod \"e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") " pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:22.995204 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:22.995151    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:23.014930 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:23.014892    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" event={"ID":"65ad5b87-68ce-4bae-8702-05416361c76f","Type":"ContainerStarted","Data":"55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5"}
Sep 10 15:00:23.288271 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:23.285952    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"]
Sep 10 15:00:23.294448 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:00:23.294387    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfe69fa7_f997_4edc_9ef9_4c7dcd96aef0.slice/crio-e0a063e2a3104b8b85a182f83d4ad92385027f4273ed22d8404aa328e05145ac WatchSource:0}: Error finding container e0a063e2a3104b8b85a182f83d4ad92385027f4273ed22d8404aa328e05145ac: Status 404 returned error can't find the container with id e0a063e2a3104b8b85a182f83d4ad92385027f4273ed22d8404aa328e05145ac
Sep 10 15:00:23.570237 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:23.570127    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:23.570403 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:23.570353    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 10 15:00:23.570450 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:23.570409    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 10 15:00:24.019252 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:24.019202    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02164c8d-05a4-4714-bfbe-8f0e8c7d3280" path="/var/lib/kubelet/pods/02164c8d-05a4-4714-bfbe-8f0e8c7d3280/volumes"
Sep 10 15:00:24.022709 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:24.022676    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9" event={"ID":"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0","Type":"ContainerStarted","Data":"d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2"}
Sep 10 15:00:24.022838 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:24.022722    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9" event={"ID":"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0","Type":"ContainerStarted","Data":"e0a063e2a3104b8b85a182f83d4ad92385027f4273ed22d8404aa328e05145ac"}
Sep 10 15:00:24.052209 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:24.052152    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9" podStartSLOduration=2.052137257 podStartE2EDuration="2.052137257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:24.049902894 +0000 UTC m=+1068.536590784" watchObservedRunningTime="2026-09-10 15:00:24.052137257 +0000 UTC m=+1068.538825148"
Sep 10 15:00:24.570961 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:24.570920    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 10 15:00:24.571156 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:24.570978    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 10 15:00:25.570288 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:25.570251    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 10 15:00:25.570649 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:25.570310    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 10 15:00:26.570812 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:26.570778    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 10 15:00:26.571251 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:26.570835    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 10 15:00:26.891601 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:26.891499    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:00:27.570711 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:27.570624    2558 patch_prober.go:28] interesting pod/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 10 15:00:27.570711 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:27.570694    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 10 15:00:28.619786 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:28.619751    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:28.620348 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:28.620162    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:28.622577 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:28.622551    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:00:32.996111 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:32.996079    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:32.996553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:32.996149    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:33.008998 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:33.008977    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:33.088706 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:33.088678    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:00:50.170428 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.170378    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"]
Sep 10 15:00:50.173587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.173568    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.176512 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.176488    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"openshift-service-ca.crt\""
Sep 10 15:00:50.176645 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.176545    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"default-dockercfg-xs8tp\""
Sep 10 15:00:50.177705 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.177685    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"kube-root-ca.crt\""
Sep 10 15:00:50.177829 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.177796    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"auth-test-model-16ad45-kserve-self-signed-certs\""
Sep 10 15:00:50.184821 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.184801    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"]
Sep 10 15:00:50.272423 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.272385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-dshm\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.272423 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.272423    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkskc\" (UniqueName: \"kubernetes.io/projected/056764fc-872d-409b-84b9-460bbab7e6dc-kube-api-access-jkskc\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.272662 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.272458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-tmp-dir\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.272662 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.272507    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056764fc-872d-409b-84b9-460bbab7e6dc-tls-certs\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.272662 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.272600    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-home\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.272662 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.272635    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-model-cache\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.373632 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.373592    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-dshm\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.373632 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.373634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkskc\" (UniqueName: \"kubernetes.io/projected/056764fc-872d-409b-84b9-460bbab7e6dc-kube-api-access-jkskc\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.373906 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.373753    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-tmp-dir\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.373906 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.373818    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056764fc-872d-409b-84b9-460bbab7e6dc-tls-certs\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.373906 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.373888    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-home\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.374062 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.373923    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-model-cache\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.374156 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.374134    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-tmp-dir\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.374289 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.374258    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-home\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.374289 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.374280    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-model-cache\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.376111 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.376080    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-dshm\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.376507 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.376484    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056764fc-872d-409b-84b9-460bbab7e6dc-tls-certs\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.383329 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.383306    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkskc\" (UniqueName: \"kubernetes.io/projected/056764fc-872d-409b-84b9-460bbab7e6dc-kube-api-access-jkskc\") pod \"auth-test-model-16ad45-kserve-697598bc95-p6gx2\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.483960 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.483919    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:00:50.771518 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:50.771492    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"]
Sep 10 15:00:50.772513 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:00:50.772484    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod056764fc_872d_409b_84b9_460bbab7e6dc.slice/crio-a62d15d13aab5521c90f6f138261b227d880435fa3a8454aa87aa8bb1991f7b4 WatchSource:0}: Error finding container a62d15d13aab5521c90f6f138261b227d880435fa3a8454aa87aa8bb1991f7b4: Status 404 returned error can't find the container with id a62d15d13aab5521c90f6f138261b227d880435fa3a8454aa87aa8bb1991f7b4
Sep 10 15:00:51.194840 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:51.194805    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2" event={"ID":"056764fc-872d-409b-84b9-460bbab7e6dc","Type":"ContainerStarted","Data":"9b3d718f72b63f393164f88f241e39fb3527884de108cc28cf38f46c170d1869"}
Sep 10 15:00:51.195247 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:51.194853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2" event={"ID":"056764fc-872d-409b-84b9-460bbab7e6dc","Type":"ContainerStarted","Data":"a62d15d13aab5521c90f6f138261b227d880435fa3a8454aa87aa8bb1991f7b4"}
Sep 10 15:00:51.212905 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:51.212856    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2" podStartSLOduration=1.212839231 podStartE2EDuration="1.212839231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:51.211770642 +0000 UTC m=+1095.698458544" watchObservedRunningTime="2026-09-10 15:00:51.212839231 +0000 UTC m=+1095.699527171"
Sep 10 15:00:55.085343 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:00:55.085305    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:01:00.484384 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:00.484336    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:01:00.484887 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:00.484462    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:01:00.496723 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:00.496697    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:01:01.271788 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:01.271752    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:01:08.511466 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.511430    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"]
Sep 10 15:01:08.514972 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.514955    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"
Sep 10 15:01:08.517764 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.517735    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:01:08.517903 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.517782    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:01:08.524895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.524871    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"]
Sep 10 15:01:08.535229 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.535189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4c7071-da74-4ab4-b318-de57f1be5749-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6621560b-26knh\" (UID: \"3c4c7071-da74-4ab4-b318-de57f1be5749\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"
Sep 10 15:01:08.636264 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.636226    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4c7071-da74-4ab4-b318-de57f1be5749-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6621560b-26knh\" (UID: \"3c4c7071-da74-4ab4-b318-de57f1be5749\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"
Sep 10 15:01:08.636792 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.636771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4c7071-da74-4ab4-b318-de57f1be5749-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6621560b-26knh\" (UID: \"3c4c7071-da74-4ab4-b318-de57f1be5749\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"
Sep 10 15:01:08.682740 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.682706    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:01:08.827684 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:08.827595    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"
Sep 10 15:01:09.103465 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:09.103429    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"]
Sep 10 15:01:09.104381 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:01:09.104354    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4c7071_da74_4ab4_b318_de57f1be5749.slice/crio-40c8470b049253ac43fd7899295ed4818cb7990a82d12fc35a2b7bac3a887180 WatchSource:0}: Error finding container 40c8470b049253ac43fd7899295ed4818cb7990a82d12fc35a2b7bac3a887180: Status 404 returned error can't find the container with id 40c8470b049253ac43fd7899295ed4818cb7990a82d12fc35a2b7bac3a887180
Sep 10 15:01:09.305640 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:09.305609    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh" event={"ID":"3c4c7071-da74-4ab4-b318-de57f1be5749","Type":"ContainerStarted","Data":"40c8470b049253ac43fd7899295ed4818cb7990a82d12fc35a2b7bac3a887180"}
Sep 10 15:01:10.313073 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:10.313040    2558 generic.go:356] "Generic (PLEG): container finished" podID="3c4c7071-da74-4ab4-b318-de57f1be5749" containerID="37bfb02f9670cac57fad41acb0adde4154e67c494205133ca373fa22b976e1c0" exitCode=0
Sep 10 15:01:10.313457 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:10.313099    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh" event={"ID":"3c4c7071-da74-4ab4-b318-de57f1be5749","Type":"ContainerDied","Data":"37bfb02f9670cac57fad41acb0adde4154e67c494205133ca373fa22b976e1c0"}
Sep 10 15:01:11.453790 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:11.453764    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"
Sep 10 15:01:11.563502 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:11.563474    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4c7071-da74-4ab4-b318-de57f1be5749-maas-api-service-ca\") pod \"3c4c7071-da74-4ab4-b318-de57f1be5749\" (UID: \"3c4c7071-da74-4ab4-b318-de57f1be5749\") "
Sep 10 15:01:11.563818 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:11.563798    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c4c7071-da74-4ab4-b318-de57f1be5749-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c4c7071-da74-4ab4-b318-de57f1be5749" (UID: "3c4c7071-da74-4ab4-b318-de57f1be5749"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:01:11.664749 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:11.664663    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c4c7071-da74-4ab4-b318-de57f1be5749-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:01:12.327573 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:12.327487    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"
Sep 10 15:01:12.327573 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:12.327495    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh" event={"ID":"3c4c7071-da74-4ab4-b318-de57f1be5749","Type":"ContainerDied","Data":"40c8470b049253ac43fd7899295ed4818cb7990a82d12fc35a2b7bac3a887180"}
Sep 10 15:01:12.327573 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:12.327537    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40c8470b049253ac43fd7899295ed4818cb7990a82d12fc35a2b7bac3a887180"
Sep 10 15:01:17.669330 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.669300    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"]
Sep 10 15:01:17.669730 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.669714    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c4c7071-da74-4ab4-b318-de57f1be5749" containerName="revoke-keys"
Sep 10 15:01:17.669770 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.669733    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c4c7071-da74-4ab4-b318-de57f1be5749" containerName="revoke-keys"
Sep 10 15:01:17.669843 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.669825    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c4c7071-da74-4ab4-b318-de57f1be5749" containerName="revoke-keys"
Sep 10 15:01:17.672859 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.672844    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.676779 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.676751    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"default-dockercfg-dfdcl\""
Sep 10 15:01:17.676925 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.676781    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"kube-root-ca.crt\""
Sep 10 15:01:17.676925 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.676804    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"openshift-service-ca.crt\""
Sep 10 15:01:17.676925 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.676789    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"auth-test-model-3f81b7-kserve-self-signed-certs\""
Sep 10 15:01:17.681436 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.681409    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"]
Sep 10 15:01:17.725056 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.725024    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rkwb\" (UniqueName: \"kubernetes.io/projected/7706a403-910f-46a0-9a0a-417f17366d53-kube-api-access-9rkwb\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.725254 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.725062    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-tmp-dir\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.725254 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.725080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-dshm\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.725254 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.725206    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7706a403-910f-46a0-9a0a-417f17366d53-tls-certs\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.725390 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.725279    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-model-cache\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.725390 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.725315    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-home\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.825902 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.825866    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-model-cache\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826049 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.825919    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-home\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826049 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.825964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rkwb\" (UniqueName: \"kubernetes.io/projected/7706a403-910f-46a0-9a0a-417f17366d53-kube-api-access-9rkwb\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826049 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.826003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-tmp-dir\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826201 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.826128    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-dshm\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826294 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.826279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7706a403-910f-46a0-9a0a-417f17366d53-tls-certs\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826404 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.826382    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-model-cache\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826479 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.826413    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-home\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.826479 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.826464    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-tmp-dir\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.828574 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.828545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-dshm\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.828873 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.828853    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7706a403-910f-46a0-9a0a-417f17366d53-tls-certs\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.835420 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.835397    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rkwb\" (UniqueName: \"kubernetes.io/projected/7706a403-910f-46a0-9a0a-417f17366d53-kube-api-access-9rkwb\") pod \"auth-test-model-3f81b7-kserve-6d68667d79-2dj6s\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:17.982663 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:17.982622    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:18.259831 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:18.259805    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"]
Sep 10 15:01:18.261935 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:01:18.261898    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7706a403_910f_46a0_9a0a_417f17366d53.slice/crio-1ef7b16ee0cea0b0ad7cdb8a956b0670b492619693392052797aad929f4b5392 WatchSource:0}: Error finding container 1ef7b16ee0cea0b0ad7cdb8a956b0670b492619693392052797aad929f4b5392: Status 404 returned error can't find the container with id 1ef7b16ee0cea0b0ad7cdb8a956b0670b492619693392052797aad929f4b5392
Sep 10 15:01:18.368169 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:18.368138    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s" event={"ID":"7706a403-910f-46a0-9a0a-417f17366d53","Type":"ContainerStarted","Data":"e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce"}
Sep 10 15:01:18.368346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:18.368178    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s" event={"ID":"7706a403-910f-46a0-9a0a-417f17366d53","Type":"ContainerStarted","Data":"1ef7b16ee0cea0b0ad7cdb8a956b0670b492619693392052797aad929f4b5392"}
Sep 10 15:01:24.477935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:24.477880    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s" podStartSLOduration=7.47786155 podStartE2EDuration="7.47786155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:01:18.394303095 +0000 UTC m=+1122.880990986" watchObservedRunningTime="2026-09-10 15:01:24.47786155 +0000 UTC m=+1128.964549440"
Sep 10 15:01:24.479619 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:24.479601    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"]
Sep 10 15:01:24.486304 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:24.486278    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6621560b-26knh"]
Sep 10 15:01:24.683030 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:24.682992    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:01:26.019681 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:26.019649    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c4c7071-da74-4ab4-b318-de57f1be5749" path="/var/lib/kubelet/pods/3c4c7071-da74-4ab4-b318-de57f1be5749/volumes"
Sep 10 15:01:27.983038 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:27.982983    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:27.983038 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:27.983048    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:27.995809 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:27.995783    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:28.451633 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:28.451602    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:01:32.827713 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:32.827682    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"]
Sep 10 15:01:39.284896 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:39.284859    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:01:47.848028 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:47.847974    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9" podUID="bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" containerName="main" containerID="cri-o://d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2" gracePeriod=45
Sep 10 15:01:48.095596 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.095572    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:01:48.136641 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.136549    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvqjt\" (UniqueName: \"kubernetes.io/projected/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-kube-api-access-tvqjt\") pod \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") "
Sep 10 15:01:48.136641 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.136613    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-model-cache\") pod \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") "
Sep 10 15:01:48.136876 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.136663    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-home\") pod \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") "
Sep 10 15:01:48.136876 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.136694    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tmp-dir\") pod \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") "
Sep 10 15:01:48.136876 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.136732    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-dshm\") pod \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") "
Sep 10 15:01:48.136876 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.136763    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tls-certs\") pod \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\" (UID: \"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0\") "
Sep 10 15:01:48.137089 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.136916    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-model-cache" (OuterVolumeSpecName: "model-cache") pod "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" (UID: "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:01:48.137205 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.137178    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" (UID: "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:01:48.137302 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.137193    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-home" (OuterVolumeSpecName: "home") pod "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" (UID: "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:01:48.137302 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.137206    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:01:48.139455 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.139410    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-kube-api-access-tvqjt" (OuterVolumeSpecName: "kube-api-access-tvqjt") pod "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" (UID: "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0"). InnerVolumeSpecName "kube-api-access-tvqjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:01:48.139455 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.139442    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-dshm" (OuterVolumeSpecName: "dshm") pod "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" (UID: "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:01:48.139727 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.139708    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" (UID: "bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:01:48.238583 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.238550    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:01:48.238583 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.238579    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:01:48.238583 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.238589    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:01:48.238822 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.238597    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:01:48.238822 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.238605    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvqjt\" (UniqueName: \"kubernetes.io/projected/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0-kube-api-access-tvqjt\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:01:48.557007 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.556972    2558 generic.go:356] "Generic (PLEG): container finished" podID="bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" containerID="d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2" exitCode=0
Sep 10 15:01:48.557007 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.557012    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"
Sep 10 15:01:48.557246 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.557064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9" event={"ID":"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0","Type":"ContainerDied","Data":"d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2"}
Sep 10 15:01:48.557246 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.557107    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9" event={"ID":"bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0","Type":"ContainerDied","Data":"e0a063e2a3104b8b85a182f83d4ad92385027f4273ed22d8404aa328e05145ac"}
Sep 10 15:01:48.557246 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.557128    2558 scope.go:117] "RemoveContainer" containerID="d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2"
Sep 10 15:01:48.566992 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.566974    2558 scope.go:117] "RemoveContainer" containerID="d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2"
Sep 10 15:01:48.567313 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:01:48.567293    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2\": container with ID starting with d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2 not found: ID does not exist" containerID="d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2"
Sep 10 15:01:48.567400 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.567326    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2"} err="failed to get container status \"d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2\": rpc error: code = NotFound desc = could not find container \"d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2\": container with ID starting with d1cd078a08fe698bc0a2c2e2559853244cf6ee2a8691a8b47574030a0989d6e2 not found: ID does not exist"
Sep 10 15:01:48.596278 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.596210    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"]
Sep 10 15:01:48.605193 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:48.605166    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6621560b/e2e-lifecycle-model-6621560b-kserve-555756c459-gz9s9"]
Sep 10 15:01:50.018780 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:50.018746    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" path="/var/lib/kubelet/pods/bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0/volumes"
Sep 10 15:01:54.373479 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:01:54.373443    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:02:10.577532 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:02:10.577445    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:02:24.100884 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:02:24.100843    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:02:40.274279 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:02:40.274244    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:02:52.683598 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:02:52.683554    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:03:05.569733 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:05.569702    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:03:07.789628 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.789595    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-e8249f"]
Sep 10 15:03:07.790026 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.789988    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" containerName="main"
Sep 10 15:03:07.790026 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.790000    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" containerName="main"
Sep 10 15:03:07.790108 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.790084    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfe69fa7-f997-4edc-9ef9-4c7dcd96aef0" containerName="main"
Sep 10 15:03:07.793122 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.793107    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-e8249f"
Sep 10 15:03:07.796066 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.796040    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mmqf2\""
Sep 10 15:03:07.800082 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.800053    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-e8249f"]
Sep 10 15:03:07.851510 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.851479    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncqnd\" (UniqueName: \"kubernetes.io/projected/0ace7396-c979-4bbc-8f3d-1ac1bc13dc80-kube-api-access-ncqnd\") pod \"test-curl-7659-e8249f\" (UID: \"0ace7396-c979-4bbc-8f3d-1ac1bc13dc80\") " pod="opendatahub/test-curl-7659-e8249f"
Sep 10 15:03:07.952174 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.952136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncqnd\" (UniqueName: \"kubernetes.io/projected/0ace7396-c979-4bbc-8f3d-1ac1bc13dc80-kube-api-access-ncqnd\") pod \"test-curl-7659-e8249f\" (UID: \"0ace7396-c979-4bbc-8f3d-1ac1bc13dc80\") " pod="opendatahub/test-curl-7659-e8249f"
Sep 10 15:03:07.961554 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:07.961520    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncqnd\" (UniqueName: \"kubernetes.io/projected/0ace7396-c979-4bbc-8f3d-1ac1bc13dc80-kube-api-access-ncqnd\") pod \"test-curl-7659-e8249f\" (UID: \"0ace7396-c979-4bbc-8f3d-1ac1bc13dc80\") " pod="opendatahub/test-curl-7659-e8249f"
Sep 10 15:03:08.104068 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:08.103969    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-e8249f"
Sep 10 15:03:08.383843 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:08.383818    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-e8249f"]
Sep 10 15:03:08.384768 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:03:08.384741    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ace7396_c979_4bbc_8f3d_1ac1bc13dc80.slice/crio-5c7cf7e6dfad059b7c4f0e6962e0911b0efe3d7ab73232ed2434a6b76b5d1093 WatchSource:0}: Error finding container 5c7cf7e6dfad059b7c4f0e6962e0911b0efe3d7ab73232ed2434a6b76b5d1093: Status 404 returned error can't find the container with id 5c7cf7e6dfad059b7c4f0e6962e0911b0efe3d7ab73232ed2434a6b76b5d1093
Sep 10 15:03:09.053194 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:09.053156    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-e8249f" event={"ID":"0ace7396-c979-4bbc-8f3d-1ac1bc13dc80","Type":"ContainerStarted","Data":"f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5"}
Sep 10 15:03:09.053570 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:09.053205    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-e8249f" event={"ID":"0ace7396-c979-4bbc-8f3d-1ac1bc13dc80","Type":"ContainerStarted","Data":"5c7cf7e6dfad059b7c4f0e6962e0911b0efe3d7ab73232ed2434a6b76b5d1093"}
Sep 10 15:03:09.072332 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:09.072282    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7659-e8249f" podStartSLOduration=2.072268363 podStartE2EDuration="2.072268363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:03:09.069636253 +0000 UTC m=+1233.556324141" watchObservedRunningTime="2026-09-10 15:03:09.072268363 +0000 UTC m=+1233.558956289"
Sep 10 15:03:09.654524 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:09.654492    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-e8249f"]
Sep 10 15:03:09.658883 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:09.658857    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-e8249f"]
Sep 10 15:03:10.059655 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:03:10.059628    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7659-e8249f" container="test-curl-7659-e8249f"
Sep 10 15:03:11.015855 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:11.015824    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"]
Sep 10 15:03:11.065367 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:11.065326    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7659-e8249f" podUID="0ace7396-c979-4bbc-8f3d-1ac1bc13dc80" containerName="test-curl-7659-e8249f" containerID="cri-o://f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5" gracePeriod=2
Sep 10 15:03:11.858398 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:11.858366    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"]
Sep 10 15:03:13.210157 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:13.210130    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-e8249f"
Sep 10 15:03:13.307463 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:13.307432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncqnd\" (UniqueName: \"kubernetes.io/projected/0ace7396-c979-4bbc-8f3d-1ac1bc13dc80-kube-api-access-ncqnd\") pod \"0ace7396-c979-4bbc-8f3d-1ac1bc13dc80\" (UID: \"0ace7396-c979-4bbc-8f3d-1ac1bc13dc80\") "
Sep 10 15:03:13.309617 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:13.309587    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ace7396-c979-4bbc-8f3d-1ac1bc13dc80-kube-api-access-ncqnd" (OuterVolumeSpecName: "kube-api-access-ncqnd") pod "0ace7396-c979-4bbc-8f3d-1ac1bc13dc80" (UID: "0ace7396-c979-4bbc-8f3d-1ac1bc13dc80"). InnerVolumeSpecName "kube-api-access-ncqnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:03:13.408367 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:13.408289    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncqnd\" (UniqueName: \"kubernetes.io/projected/0ace7396-c979-4bbc-8f3d-1ac1bc13dc80-kube-api-access-ncqnd\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:14.019039 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:14.019008    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ace7396-c979-4bbc-8f3d-1ac1bc13dc80" path="/var/lib/kubelet/pods/0ace7396-c979-4bbc-8f3d-1ac1bc13dc80/volumes"
Sep 10 15:03:14.085298 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:14.085262    2558 generic.go:356] "Generic (PLEG): container finished" podID="0ace7396-c979-4bbc-8f3d-1ac1bc13dc80" containerID="f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5" exitCode=137
Sep 10 15:03:14.085483 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:14.085305    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-e8249f"
Sep 10 15:03:14.085483 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:14.085370    2558 scope.go:117] "RemoveContainer" containerID="f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5"
Sep 10 15:03:14.097593 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:14.097573    2558 scope.go:117] "RemoveContainer" containerID="f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5"
Sep 10 15:03:14.097914 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:03:14.097892    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5\": container with ID starting with f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5 not found: ID does not exist" containerID="f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5"
Sep 10 15:03:14.097985 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:14.097927    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5"} err="failed to get container status \"f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5\": rpc error: code = NotFound desc = could not find container \"f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5\": container with ID starting with f82d95b53b43db9e44b964f121d38c822cf65d8e9e86feeda71356a03e17abd5 not found: ID does not exist"
Sep 10 15:03:23.081300 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:23.081258    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:03:26.035814 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.035778    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2" podUID="056764fc-872d-409b-84b9-460bbab7e6dc" containerName="main" containerID="cri-o://9b3d718f72b63f393164f88f241e39fb3527884de108cc28cf38f46c170d1869" gracePeriod=45
Sep 10 15:03:26.163528 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.163491    2558 generic.go:356] "Generic (PLEG): container finished" podID="056764fc-872d-409b-84b9-460bbab7e6dc" containerID="9b3d718f72b63f393164f88f241e39fb3527884de108cc28cf38f46c170d1869" exitCode=0
Sep 10 15:03:26.163709 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.163569    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2" event={"ID":"056764fc-872d-409b-84b9-460bbab7e6dc","Type":"ContainerDied","Data":"9b3d718f72b63f393164f88f241e39fb3527884de108cc28cf38f46c170d1869"}
Sep 10 15:03:26.299243 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.299166    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:03:26.441955 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.441915    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056764fc-872d-409b-84b9-460bbab7e6dc-tls-certs\") pod \"056764fc-872d-409b-84b9-460bbab7e6dc\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") "
Sep 10 15:03:26.441955 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.441960    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-tmp-dir\") pod \"056764fc-872d-409b-84b9-460bbab7e6dc\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") "
Sep 10 15:03:26.442158 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442008    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-home\") pod \"056764fc-872d-409b-84b9-460bbab7e6dc\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") "
Sep 10 15:03:26.442158 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442048    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-model-cache\") pod \"056764fc-872d-409b-84b9-460bbab7e6dc\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") "
Sep 10 15:03:26.442158 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442096    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkskc\" (UniqueName: \"kubernetes.io/projected/056764fc-872d-409b-84b9-460bbab7e6dc-kube-api-access-jkskc\") pod \"056764fc-872d-409b-84b9-460bbab7e6dc\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") "
Sep 10 15:03:26.442158 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442124    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-dshm\") pod \"056764fc-872d-409b-84b9-460bbab7e6dc\" (UID: \"056764fc-872d-409b-84b9-460bbab7e6dc\") "
Sep 10 15:03:26.442413 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442318    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-home" (OuterVolumeSpecName: "home") pod "056764fc-872d-409b-84b9-460bbab7e6dc" (UID: "056764fc-872d-409b-84b9-460bbab7e6dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:26.442413 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442356    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "056764fc-872d-409b-84b9-460bbab7e6dc" (UID: "056764fc-872d-409b-84b9-460bbab7e6dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:26.442525 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442411    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "056764fc-872d-409b-84b9-460bbab7e6dc" (UID: "056764fc-872d-409b-84b9-460bbab7e6dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:26.442582 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442562    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:26.442669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442589    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:26.442669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.442602    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:26.444465 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.444433    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056764fc-872d-409b-84b9-460bbab7e6dc-kube-api-access-jkskc" (OuterVolumeSpecName: "kube-api-access-jkskc") pod "056764fc-872d-409b-84b9-460bbab7e6dc" (UID: "056764fc-872d-409b-84b9-460bbab7e6dc"). InnerVolumeSpecName "kube-api-access-jkskc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:03:26.444575 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.444437    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/056764fc-872d-409b-84b9-460bbab7e6dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "056764fc-872d-409b-84b9-460bbab7e6dc" (UID: "056764fc-872d-409b-84b9-460bbab7e6dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:03:26.444575 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.444461    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-dshm" (OuterVolumeSpecName: "dshm") pod "056764fc-872d-409b-84b9-460bbab7e6dc" (UID: "056764fc-872d-409b-84b9-460bbab7e6dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:26.543933 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.543898    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/056764fc-872d-409b-84b9-460bbab7e6dc-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:26.543933 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.543929    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkskc\" (UniqueName: \"kubernetes.io/projected/056764fc-872d-409b-84b9-460bbab7e6dc-kube-api-access-jkskc\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:26.543933 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.543938    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/056764fc-872d-409b-84b9-460bbab7e6dc-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:26.878564 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:26.878525    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s" podUID="7706a403-910f-46a0-9a0a-417f17366d53" containerName="main" containerID="cri-o://e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce" gracePeriod=45
Sep 10 15:03:27.122952 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.122926    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:03:27.149340 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149263    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-home\") pod \"7706a403-910f-46a0-9a0a-417f17366d53\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") "
Sep 10 15:03:27.149340 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149304    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-dshm\") pod \"7706a403-910f-46a0-9a0a-417f17366d53\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") "
Sep 10 15:03:27.149340 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149336    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rkwb\" (UniqueName: \"kubernetes.io/projected/7706a403-910f-46a0-9a0a-417f17366d53-kube-api-access-9rkwb\") pod \"7706a403-910f-46a0-9a0a-417f17366d53\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") "
Sep 10 15:03:27.149616 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149351    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-tmp-dir\") pod \"7706a403-910f-46a0-9a0a-417f17366d53\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") "
Sep 10 15:03:27.149616 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149370    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-model-cache\") pod \"7706a403-910f-46a0-9a0a-417f17366d53\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") "
Sep 10 15:03:27.149616 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149398    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7706a403-910f-46a0-9a0a-417f17366d53-tls-certs\") pod \"7706a403-910f-46a0-9a0a-417f17366d53\" (UID: \"7706a403-910f-46a0-9a0a-417f17366d53\") "
Sep 10 15:03:27.149616 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149518    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-home" (OuterVolumeSpecName: "home") pod "7706a403-910f-46a0-9a0a-417f17366d53" (UID: "7706a403-910f-46a0-9a0a-417f17366d53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:27.149825 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149642    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7706a403-910f-46a0-9a0a-417f17366d53" (UID: "7706a403-910f-46a0-9a0a-417f17366d53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:27.149825 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149651    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-model-cache" (OuterVolumeSpecName: "model-cache") pod "7706a403-910f-46a0-9a0a-417f17366d53" (UID: "7706a403-910f-46a0-9a0a-417f17366d53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:27.149932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149831    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:27.149932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149860    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:27.149932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.149876    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:27.152310 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.152281    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-dshm" (OuterVolumeSpecName: "dshm") pod "7706a403-910f-46a0-9a0a-417f17366d53" (UID: "7706a403-910f-46a0-9a0a-417f17366d53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:03:27.152431 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.152321    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7706a403-910f-46a0-9a0a-417f17366d53-kube-api-access-9rkwb" (OuterVolumeSpecName: "kube-api-access-9rkwb") pod "7706a403-910f-46a0-9a0a-417f17366d53" (UID: "7706a403-910f-46a0-9a0a-417f17366d53"). InnerVolumeSpecName "kube-api-access-9rkwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:03:27.152431 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.152392    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7706a403-910f-46a0-9a0a-417f17366d53-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7706a403-910f-46a0-9a0a-417f17366d53" (UID: "7706a403-910f-46a0-9a0a-417f17366d53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:03:27.172674 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.172647    2558 generic.go:356] "Generic (PLEG): container finished" podID="7706a403-910f-46a0-9a0a-417f17366d53" containerID="e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce" exitCode=0
Sep 10 15:03:27.172801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.172689    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"
Sep 10 15:03:27.172801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.172732    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s" event={"ID":"7706a403-910f-46a0-9a0a-417f17366d53","Type":"ContainerDied","Data":"e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce"}
Sep 10 15:03:27.172801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.172772    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s" event={"ID":"7706a403-910f-46a0-9a0a-417f17366d53","Type":"ContainerDied","Data":"1ef7b16ee0cea0b0ad7cdb8a956b0670b492619693392052797aad929f4b5392"}
Sep 10 15:03:27.172801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.172791    2558 scope.go:117] "RemoveContainer" containerID="e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce"
Sep 10 15:03:27.174449 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.174378    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2" event={"ID":"056764fc-872d-409b-84b9-460bbab7e6dc","Type":"ContainerDied","Data":"a62d15d13aab5521c90f6f138261b227d880435fa3a8454aa87aa8bb1991f7b4"}
Sep 10 15:03:27.174449 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.174412    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"
Sep 10 15:03:27.183654 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.183592    2558 scope.go:117] "RemoveContainer" containerID="e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce"
Sep 10 15:03:27.183999 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:03:27.183976    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce\": container with ID starting with e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce not found: ID does not exist" containerID="e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce"
Sep 10 15:03:27.184097 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.184011    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce"} err="failed to get container status \"e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce\": rpc error: code = NotFound desc = could not find container \"e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce\": container with ID starting with e7beebfcca9f8bf337c5c0549b84addab9e31c294c4268351c05b7860c70b7ce not found: ID does not exist"
Sep 10 15:03:27.184097 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.184033    2558 scope.go:117] "RemoveContainer" containerID="9b3d718f72b63f393164f88f241e39fb3527884de108cc28cf38f46c170d1869"
Sep 10 15:03:27.198750 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.198725    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"]
Sep 10 15:03:27.204356 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.204335    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/auth-test-model-16ad45-kserve-697598bc95-p6gx2"]
Sep 10 15:03:27.216998 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.216973    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"]
Sep 10 15:03:27.222639 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.222617    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/auth-test-model-3f81b7-kserve-6d68667d79-2dj6s"]
Sep 10 15:03:27.250411 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.250380    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7706a403-910f-46a0-9a0a-417f17366d53-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:27.250411 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.250407    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rkwb\" (UniqueName: \"kubernetes.io/projected/7706a403-910f-46a0-9a0a-417f17366d53-kube-api-access-9rkwb\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:27.250561 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:27.250425    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7706a403-910f-46a0-9a0a-417f17366d53-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:03:28.019665 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:28.019632    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="056764fc-872d-409b-84b9-460bbab7e6dc" path="/var/lib/kubelet/pods/056764fc-872d-409b-84b9-460bbab7e6dc/volumes"
Sep 10 15:03:28.020046 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:28.020033    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7706a403-910f-46a0-9a0a-417f17366d53" path="/var/lib/kubelet/pods/7706a403-910f-46a0-9a0a-417f17366d53/volumes"
Sep 10 15:03:33.787865 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.787828    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"]
Sep 10 15:03:33.788309 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788279    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ace7396-c979-4bbc-8f3d-1ac1bc13dc80" containerName="test-curl-7659-e8249f"
Sep 10 15:03:33.788309 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788295    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ace7396-c979-4bbc-8f3d-1ac1bc13dc80" containerName="test-curl-7659-e8249f"
Sep 10 15:03:33.788309 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788306    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7706a403-910f-46a0-9a0a-417f17366d53" containerName="main"
Sep 10 15:03:33.788309 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788312    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7706a403-910f-46a0-9a0a-417f17366d53" containerName="main"
Sep 10 15:03:33.788458 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788328    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056764fc-872d-409b-84b9-460bbab7e6dc" containerName="main"
Sep 10 15:03:33.788458 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788334    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="056764fc-872d-409b-84b9-460bbab7e6dc" containerName="main"
Sep 10 15:03:33.788458 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788405    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ace7396-c979-4bbc-8f3d-1ac1bc13dc80" containerName="test-curl-7659-e8249f"
Sep 10 15:03:33.788458 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788416    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="056764fc-872d-409b-84b9-460bbab7e6dc" containerName="main"
Sep 10 15:03:33.788458 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.788422    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7706a403-910f-46a0-9a0a-417f17366d53" containerName="main"
Sep 10 15:03:33.791528 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.791512    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.795421 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.795388    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"kube-root-ca.crt\""
Sep 10 15:03:33.795421 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.795411    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"sub-test-model-16ad45-kserve-self-signed-certs\""
Sep 10 15:03:33.795639 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.795413    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"openshift-service-ca.crt\""
Sep 10 15:03:33.795639 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.795398    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"default-dockercfg-xs8tp\""
Sep 10 15:03:33.804698 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.804670    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tls-certs\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.804828 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.804715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-model-cache\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.804828 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.804758    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tmp-dir\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.804916 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.804878    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-home\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.804916 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.804899    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-dshm\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.804995 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.804947    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7wjm\" (UniqueName: \"kubernetes.io/projected/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-kube-api-access-m7wjm\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.806124 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.806103    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"]
Sep 10 15:03:33.905766 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.905726    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tls-certs\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.905766 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.905769    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-model-cache\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.906009 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.905796    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tmp-dir\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.906009 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.905871    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-home\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.906009 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.905891    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-dshm\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.906009 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.905925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7wjm\" (UniqueName: \"kubernetes.io/projected/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-kube-api-access-m7wjm\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.906260 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.906189    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-model-cache\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.906334 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.906294    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tmp-dir\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.906386 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.906342    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-home\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.908258 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.908233    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-dshm\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.908539 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.908522    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tls-certs\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:33.915895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:33.915864    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7wjm\" (UniqueName: \"kubernetes.io/projected/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-kube-api-access-m7wjm\") pod \"sub-test-model-16ad45-kserve-779d6cd9cc-9k9st\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:34.101051 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:34.100961    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:34.379371 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:34.379338    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"]
Sep 10 15:03:34.379857 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:03:34.379837    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e5f8bf1_b7f0_43f9_93b1_29ed57127215.slice/crio-64ce1db7cfc83b5256812044668d85fbcaa6d210f438a531b5a9c2a3b1e8350e WatchSource:0}: Error finding container 64ce1db7cfc83b5256812044668d85fbcaa6d210f438a531b5a9c2a3b1e8350e: Status 404 returned error can't find the container with id 64ce1db7cfc83b5256812044668d85fbcaa6d210f438a531b5a9c2a3b1e8350e
Sep 10 15:03:35.226502 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:35.226470    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st" event={"ID":"7e5f8bf1-b7f0-43f9-93b1-29ed57127215","Type":"ContainerStarted","Data":"43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6"}
Sep 10 15:03:35.226871 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:35.226508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st" event={"ID":"7e5f8bf1-b7f0-43f9-93b1-29ed57127215","Type":"ContainerStarted","Data":"64ce1db7cfc83b5256812044668d85fbcaa6d210f438a531b5a9c2a3b1e8350e"}
Sep 10 15:03:35.248311 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:35.248253    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st" podStartSLOduration=2.248236041 podStartE2EDuration="2.248236041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:03:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:03:35.245525018 +0000 UTC m=+1259.732212909" watchObservedRunningTime="2026-09-10 15:03:35.248236041 +0000 UTC m=+1259.734923924"
Sep 10 15:03:41.995357 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:41.995325    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"]
Sep 10 15:03:42.000259 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.000212    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.003504 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.003484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-cb65dc-openshift-default-dockercfg-dr6tj\""
Sep 10 15:03:42.019073 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.019051    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"]
Sep 10 15:03:42.084579 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.084549    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:03:42.088397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088369    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-podinfo\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088552 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088406    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-envoy\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088552 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088471    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-token\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088729 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088546    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-credential-socket\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088729 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088602    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0edab5d4-d7f4-401f-8176-2f624de66c00-istiod-ca-cert\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088729 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-socket\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088729 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9vdg\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-kube-api-access-d9vdg\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088729 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088725    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-certs\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.088997 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.088779    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-data\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190128 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-certs\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190319 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190154    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-data\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190319 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190184    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-podinfo\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190319 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190249    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-envoy\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190319 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190316    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-token\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190580 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190369    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-credential-socket\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190580 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190430    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0edab5d4-d7f4-401f-8176-2f624de66c00-istiod-ca-cert\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190580 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190458    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-socket\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190580 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9vdg\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-kube-api-access-d9vdg\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190792 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190585    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-certs\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.190792 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-data\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.191006 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190980    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-socket\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.191119 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.190997    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-credential-socket\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.191508 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.191485    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0edab5d4-d7f4-401f-8176-2f624de66c00-istiod-ca-cert\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.192975 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.192942    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-envoy\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.193460 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.193437    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-podinfo\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.199251 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.199195    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-token\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.199669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.199648    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9vdg\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-kube-api-access-d9vdg\") pod \"e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") " pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.310070 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.309983    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:42.598992 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.598944    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"]
Sep 10 15:03:42.602030 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:03:42.602004    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0edab5d4_d7f4_401f_8176_2f624de66c00.slice/crio-8274d1a41a51ce8748dec9a703c71fa811e0458e01938b636a67f1ad82a80b4d WatchSource:0}: Error finding container 8274d1a41a51ce8748dec9a703c71fa811e0458e01938b636a67f1ad82a80b4d: Status 404 returned error can't find the container with id 8274d1a41a51ce8748dec9a703c71fa811e0458e01938b636a67f1ad82a80b4d
Sep 10 15:03:42.605041 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.605000    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:03:42.605137 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.605091    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:03:42.605176 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:42.605135    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:03:43.281008 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:43.280973    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" event={"ID":"0edab5d4-d7f4-401f-8176-2f624de66c00","Type":"ContainerStarted","Data":"2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54"}
Sep 10 15:03:43.281416 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:43.281034    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" event={"ID":"0edab5d4-d7f4-401f-8176-2f624de66c00","Type":"ContainerStarted","Data":"8274d1a41a51ce8748dec9a703c71fa811e0458e01938b636a67f1ad82a80b4d"}
Sep 10 15:03:43.309680 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:43.309632    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podStartSLOduration=2.30961656 podStartE2EDuration="2.30961656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:03:43.308138756 +0000 UTC m=+1267.794826649" watchObservedRunningTime="2026-09-10 15:03:43.30961656 +0000 UTC m=+1267.796304450"
Sep 10 15:03:43.310134 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:43.310115    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:43.315373 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:43.315353    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:44.101711 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:44.101675    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:44.101922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:44.101748    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:44.114891 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:44.114871    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:44.287788 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:44.287744    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:44.288577 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:44.288555    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:03:44.299183 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:44.299156    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:03:55.182364 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:03:55.182327    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:04:02.201243 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.200997    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"]
Sep 10 15:04:02.211075 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.211044    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.217378 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.217350    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"default-dockercfg-dfdcl\""
Sep 10 15:04:02.217655 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.217444    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"kube-root-ca.crt\""
Sep 10 15:04:02.217746 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.217517    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"openshift-service-ca.crt\""
Sep 10 15:04:02.217746 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.217539    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-3f81b7\"/\"sub-test-model-3f81b7-kserve-self-signed-certs\""
Sep 10 15:04:02.226440 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.226416    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"]
Sep 10 15:04:02.309204 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.309164    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5840df-93c3-420b-903a-425f04aca384-tls-certs\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.309204 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.309233    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-home\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.309462 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.309284    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-model-cache\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.309462 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.309312    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-dshm\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.309462 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.309363    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-tmp-dir\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.309462 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.309385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gnwj\" (UniqueName: \"kubernetes.io/projected/df5840df-93c3-420b-903a-425f04aca384-kube-api-access-9gnwj\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.410931 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.410897    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-tmp-dir\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411126 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.410942    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gnwj\" (UniqueName: \"kubernetes.io/projected/df5840df-93c3-420b-903a-425f04aca384-kube-api-access-9gnwj\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411126 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.410975    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5840df-93c3-420b-903a-425f04aca384-tls-certs\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411126 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.411023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-home\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411126 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.411057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-model-cache\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411126 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.411107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-dshm\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411461 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.411440    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-home\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.411527    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-model-cache\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.411674 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.411609    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-tmp-dir\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.413563 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.413535    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-dshm\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.413826 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.413809    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5840df-93c3-420b-903a-425f04aca384-tls-certs\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.421059 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.421038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gnwj\" (UniqueName: \"kubernetes.io/projected/df5840df-93c3-420b-903a-425f04aca384-kube-api-access-9gnwj\") pod \"sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") " pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.521893 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.521799    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:02.815166 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:02.815140    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"]
Sep 10 15:04:02.817760 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:04:02.817720    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf5840df_93c3_420b_903a_425f04aca384.slice/crio-dd4a22afee7af64034a3752c7926ad87e83155616aaa7f8964aa3fdc20bcba07 WatchSource:0}: Error finding container dd4a22afee7af64034a3752c7926ad87e83155616aaa7f8964aa3fdc20bcba07: Status 404 returned error can't find the container with id dd4a22afee7af64034a3752c7926ad87e83155616aaa7f8964aa3fdc20bcba07
Sep 10 15:04:03.429791 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:03.429751    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4" event={"ID":"df5840df-93c3-420b-903a-425f04aca384","Type":"ContainerStarted","Data":"e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd"}
Sep 10 15:04:03.430235 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:03.429805    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4" event={"ID":"df5840df-93c3-420b-903a-425f04aca384","Type":"ContainerStarted","Data":"dd4a22afee7af64034a3752c7926ad87e83155616aaa7f8964aa3fdc20bcba07"}
Sep 10 15:04:03.460669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:03.460616    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4" podStartSLOduration=1.46059852 podStartE2EDuration="1.46059852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:04:03.458607239 +0000 UTC m=+1287.945295134" watchObservedRunningTime="2026-09-10 15:04:03.46059852 +0000 UTC m=+1287.947286411"
Sep 10 15:04:06.451174 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:06.451141    2558 generic.go:356] "Generic (PLEG): container finished" podID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerID="2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54" exitCode=0
Sep 10 15:04:06.451607 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:06.451240    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" event={"ID":"0edab5d4-d7f4-401f-8176-2f624de66c00","Type":"ContainerDied","Data":"2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54"}
Sep 10 15:04:06.451699 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:06.451676    2558 scope.go:117] "RemoveContainer" containerID="2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54"
Sep 10 15:04:06.452886 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:06.452854    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:04:06.452986 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:06.452957    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:04:06.453025 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:06.452994    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:04:07.459602 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:07.459566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" event={"ID":"0edab5d4-d7f4-401f-8176-2f624de66c00","Type":"ContainerStarted","Data":"ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e"}
Sep 10 15:04:08.310649 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:08.310607    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:04:08.310865 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:08.310831    2558 patch_prober.go:28] interesting pod/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 10 15:04:08.310947 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:08.310904    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 10 15:04:09.000503 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:09.000364    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:04:09.311002 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:09.310913    2558 patch_prober.go:28] interesting pod/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 10 15:04:09.311002 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:09.310979    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 10 15:04:10.310628 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:10.310590    2558 patch_prober.go:28] interesting pod/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 10 15:04:10.311073 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:10.310659    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 10 15:04:11.311119 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:11.311087    2558 patch_prober.go:28] interesting pod/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 10 15:04:11.311598 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:11.311144    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 10 15:04:12.311049 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:12.311017    2558 patch_prober.go:28] interesting pod/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 10 15:04:12.311261 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:12.311085    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 10 15:04:12.521979 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:12.521951    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:12.522188 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:12.522177    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:12.534632 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:12.534608    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:13.208644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.208613    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"]
Sep 10 15:04:13.211185 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.211170    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"
Sep 10 15:04:13.213875 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.213851    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:04:13.213875 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.213853    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:04:13.220943 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.220921    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"]
Sep 10 15:04:13.314746 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.314718    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:04:13.315205 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.315156    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:04:13.315933 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.315915    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:04:13.333354 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.333328    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8dfef-3b91-49e9-9121-58fd1d753f51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-550dc3-vblrh\" (UID: \"bbd8dfef-3b91-49e9-9121-58fd1d753f51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"
Sep 10 15:04:13.433849 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.433815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8dfef-3b91-49e9-9121-58fd1d753f51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-550dc3-vblrh\" (UID: \"bbd8dfef-3b91-49e9-9121-58fd1d753f51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"
Sep 10 15:04:13.434428 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.434404    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8dfef-3b91-49e9-9121-58fd1d753f51-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-550dc3-vblrh\" (UID: \"bbd8dfef-3b91-49e9-9121-58fd1d753f51\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"
Sep 10 15:04:13.514754 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.514667    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:04:13.522232 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.522204    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"
Sep 10 15:04:13.801201 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:13.801165    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"]
Sep 10 15:04:13.802667 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:04:13.802642    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd8dfef_3b91_49e9_9121_58fd1d753f51.slice/crio-196762402f140fb84227e689369d760daf213f3e6b431428f38cb963aef628bf WatchSource:0}: Error finding container 196762402f140fb84227e689369d760daf213f3e6b431428f38cb963aef628bf: Status 404 returned error can't find the container with id 196762402f140fb84227e689369d760daf213f3e6b431428f38cb963aef628bf
Sep 10 15:04:14.511165 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:14.511126    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh" event={"ID":"bbd8dfef-3b91-49e9-9121-58fd1d753f51","Type":"ContainerStarted","Data":"010a12e6be097d2366a157a0591147a1d18310513418340d67b9ec62085666ff"}
Sep 10 15:04:14.511644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:14.511212    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh" event={"ID":"bbd8dfef-3b91-49e9-9121-58fd1d753f51","Type":"ContainerStarted","Data":"196762402f140fb84227e689369d760daf213f3e6b431428f38cb963aef628bf"}
Sep 10 15:04:14.530764 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:14.530715    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh" podStartSLOduration=1.530700102 podStartE2EDuration="1.530700102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:04:14.528544648 +0000 UTC m=+1299.015232538" watchObservedRunningTime="2026-09-10 15:04:14.530700102 +0000 UTC m=+1299.017387993"
Sep 10 15:04:15.519389 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:15.519356    2558 generic.go:356] "Generic (PLEG): container finished" podID="bbd8dfef-3b91-49e9-9121-58fd1d753f51" containerID="010a12e6be097d2366a157a0591147a1d18310513418340d67b9ec62085666ff" exitCode=0
Sep 10 15:04:15.519784 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:15.519451    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh" event={"ID":"bbd8dfef-3b91-49e9-9121-58fd1d753f51","Type":"ContainerDied","Data":"010a12e6be097d2366a157a0591147a1d18310513418340d67b9ec62085666ff"}
Sep 10 15:04:16.664597 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:16.664572    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"
Sep 10 15:04:16.769865 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:16.769824    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8dfef-3b91-49e9-9121-58fd1d753f51-maas-api-service-ca\") pod \"bbd8dfef-3b91-49e9-9121-58fd1d753f51\" (UID: \"bbd8dfef-3b91-49e9-9121-58fd1d753f51\") "
Sep 10 15:04:16.770208 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:16.770184    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8dfef-3b91-49e9-9121-58fd1d753f51-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbd8dfef-3b91-49e9-9121-58fd1d753f51" (UID: "bbd8dfef-3b91-49e9-9121-58fd1d753f51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:04:16.870924 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:16.870833    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8dfef-3b91-49e9-9121-58fd1d753f51-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:17.532747 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:17.532710    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"
Sep 10 15:04:17.532921 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:17.532742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh" event={"ID":"bbd8dfef-3b91-49e9-9121-58fd1d753f51","Type":"ContainerDied","Data":"196762402f140fb84227e689369d760daf213f3e6b431428f38cb963aef628bf"}
Sep 10 15:04:17.532921 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:17.532786    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="196762402f140fb84227e689369d760daf213f3e6b431428f38cb963aef628bf"
Sep 10 15:04:28.989021 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:28.988989    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"]
Sep 10 15:04:28.993258 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:28.993233    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-550dc3-vblrh"]
Sep 10 15:04:30.020523 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:30.020491    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd8dfef-3b91-49e9-9121-58fd1d753f51" path="/var/lib/kubelet/pods/bbd8dfef-3b91-49e9-9121-58fd1d753f51/volumes"
Sep 10 15:04:32.989926 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:32.989894    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:04:36.129016 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:36.128983    2558 scope.go:117] "RemoveContainer" containerID="557aa338de0e0b66e31fd2bc50ffbec5cf3d3cc567bc318d0108eb45ae5dcd51"
Sep 10 15:04:49.772937 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:49.772900    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:04:51.728606 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.728571    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-74cf8a"]
Sep 10 15:04:51.729210 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.729194    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8dfef-3b91-49e9-9121-58fd1d753f51" containerName="revoke-keys"
Sep 10 15:04:51.729210 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.729213    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8dfef-3b91-49e9-9121-58fd1d753f51" containerName="revoke-keys"
Sep 10 15:04:51.729352 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.729341    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd8dfef-3b91-49e9-9121-58fd1d753f51" containerName="revoke-keys"
Sep 10 15:04:51.734374 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.734349    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:51.738016 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.737990    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mmqf2\""
Sep 10 15:04:51.739298 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.739279    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-74cf8a"]
Sep 10 15:04:51.842035 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.841999    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn28p\" (UniqueName: \"kubernetes.io/projected/82940ddd-71f3-4127-9a00-473533b16bba-kube-api-access-kn28p\") pod \"test-curl-7659-74cf8a\" (UID: \"82940ddd-71f3-4127-9a00-473533b16bba\") " pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:51.942971 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.942933    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn28p\" (UniqueName: \"kubernetes.io/projected/82940ddd-71f3-4127-9a00-473533b16bba-kube-api-access-kn28p\") pod \"test-curl-7659-74cf8a\" (UID: \"82940ddd-71f3-4127-9a00-473533b16bba\") " pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:51.951612 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:51.951582    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn28p\" (UniqueName: \"kubernetes.io/projected/82940ddd-71f3-4127-9a00-473533b16bba-kube-api-access-kn28p\") pod \"test-curl-7659-74cf8a\" (UID: \"82940ddd-71f3-4127-9a00-473533b16bba\") " pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:52.051973 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:52.051889    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:52.328135 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:52.328106    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-74cf8a"]
Sep 10 15:04:52.329972 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:04:52.329941    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82940ddd_71f3_4127_9a00_473533b16bba.slice/crio-68f2d082dbc1081851f7332e9c3b119b0904376c576f4314974525435d8b263f WatchSource:0}: Error finding container 68f2d082dbc1081851f7332e9c3b119b0904376c576f4314974525435d8b263f: Status 404 returned error can't find the container with id 68f2d082dbc1081851f7332e9c3b119b0904376c576f4314974525435d8b263f
Sep 10 15:04:52.757647 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:52.757610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-74cf8a" event={"ID":"82940ddd-71f3-4127-9a00-473533b16bba","Type":"ContainerStarted","Data":"32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a"}
Sep 10 15:04:52.758076 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:52.757656    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-74cf8a" event={"ID":"82940ddd-71f3-4127-9a00-473533b16bba","Type":"ContainerStarted","Data":"68f2d082dbc1081851f7332e9c3b119b0904376c576f4314974525435d8b263f"}
Sep 10 15:04:52.775681 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:52.775635    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7659-74cf8a" podStartSLOduration=1.775619335 podStartE2EDuration="1.775619335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:04:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:04:52.774483557 +0000 UTC m=+1337.261171459" watchObservedRunningTime="2026-09-10 15:04:52.775619335 +0000 UTC m=+1337.262307226"
Sep 10 15:04:53.353474 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:53.353438    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-74cf8a"]
Sep 10 15:04:53.356294 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:53.356264    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-74cf8a"]
Sep 10 15:04:53.765080 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:04:53.765049    2558 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7659-74cf8a" container="test-curl-7659-74cf8a"
Sep 10 15:04:54.771235 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:54.771191    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7659-74cf8a" podUID="82940ddd-71f3-4127-9a00-473533b16bba" containerName="test-curl-7659-74cf8a" containerID="cri-o://32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a" gracePeriod=2
Sep 10 15:04:56.963166 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:56.963141    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:57.095724 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.095624    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn28p\" (UniqueName: \"kubernetes.io/projected/82940ddd-71f3-4127-9a00-473533b16bba-kube-api-access-kn28p\") pod \"82940ddd-71f3-4127-9a00-473533b16bba\" (UID: \"82940ddd-71f3-4127-9a00-473533b16bba\") "
Sep 10 15:04:57.097906 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.097880    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82940ddd-71f3-4127-9a00-473533b16bba-kube-api-access-kn28p" (OuterVolumeSpecName: "kube-api-access-kn28p") pod "82940ddd-71f3-4127-9a00-473533b16bba" (UID: "82940ddd-71f3-4127-9a00-473533b16bba"). InnerVolumeSpecName "kube-api-access-kn28p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:04:57.196674 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.196629    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn28p\" (UniqueName: \"kubernetes.io/projected/82940ddd-71f3-4127-9a00-473533b16bba-kube-api-access-kn28p\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:04:57.790275 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.790236    2558 generic.go:356] "Generic (PLEG): container finished" podID="82940ddd-71f3-4127-9a00-473533b16bba" containerID="32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a" exitCode=137
Sep 10 15:04:57.790275 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.790275    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:57.790535 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.790333    2558 scope.go:117] "RemoveContainer" containerID="32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a"
Sep 10 15:04:57.800078 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.800061    2558 scope.go:117] "RemoveContainer" containerID="32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a"
Sep 10 15:04:57.800451 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:04:57.800430    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a\": container with ID starting with 32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a not found: ID does not exist" containerID="32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a"
Sep 10 15:04:57.800520 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.800462    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a"} err="failed to get container status \"32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a\": rpc error: code = NotFound desc = could not find container \"32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a\": container with ID starting with 32cc59461b6e01bbfc9da1d5bbf329f2fa3b9c476ee197ef6171725c95ee361a not found: ID does not exist"
Sep 10 15:04:57.800520 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:04:57.800492    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-74cf8a\" is forbidden: User \"system:node:ip-10-0-142-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-253.ec2.internal' and this object" podUID="82940ddd-71f3-4127-9a00-473533b16bba" pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:57.916241 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:57.916187    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"]
Sep 10 15:04:57.937757 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:04:57.937715    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-74cf8a\" is forbidden: User \"system:node:ip-10-0-142-253.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-253.ec2.internal' and this object" podUID="82940ddd-71f3-4127-9a00-473533b16bba" pod="opendatahub/test-curl-7659-74cf8a"
Sep 10 15:04:58.018883 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:58.018848    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82940ddd-71f3-4127-9a00-473533b16bba" path="/var/lib/kubelet/pods/82940ddd-71f3-4127-9a00-473533b16bba/volumes"
Sep 10 15:04:58.116187 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:58.116096    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"]
Sep 10 15:04:58.116366 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:58.116344    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy" containerID="cri-o://ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e" gracePeriod=30
Sep 10 15:04:58.317583 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:58.317547    2558 patch_prober.go:28] interesting pod/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 15:04:58.317737 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:58.317617    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 15:04:59.166182 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:04:59.166145    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"]
Sep 10 15:05:03.364290 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.364265    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:05:03.560506 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560470    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-socket\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560676 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560527    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9vdg\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-kube-api-access-d9vdg\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560676 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560564    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-certs\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560676 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560597    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-token\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560676 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560664    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-envoy\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560743    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-data\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560789    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-credential-socket\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560863    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-podinfo\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.560922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.560886    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0edab5d4-d7f4-401f-8176-2f624de66c00-istiod-ca-cert\") pod \"0edab5d4-d7f4-401f-8176-2f624de66c00\" (UID: \"0edab5d4-d7f4-401f-8176-2f624de66c00\") "
Sep 10 15:05:03.561081 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561032    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-data" (OuterVolumeSpecName: "istio-data") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:03.561144 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561094    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:03.561144 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561105    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:03.561284 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561257    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:03.561377 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561345    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.561377 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561371    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-data\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.561573 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561378    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0edab5d4-d7f4-401f-8176-2f624de66c00-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:05:03.561573 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561388    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-credential-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.561573 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.561403    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-workload-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.563101 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.563077    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-kube-api-access-d9vdg" (OuterVolumeSpecName: "kube-api-access-d9vdg") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "kube-api-access-d9vdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:05:03.563280 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.563258    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:03.563785 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.563767    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 15:05:03.563854 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.563795    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-token" (OuterVolumeSpecName: "istio-token") pod "0edab5d4-d7f4-401f-8176-2f624de66c00" (UID: "0edab5d4-d7f4-401f-8176-2f624de66c00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:05:03.585027 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.585001    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:05:03.662030 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.661996    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-podinfo\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.662030 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.662028    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0edab5d4-d7f4-401f-8176-2f624de66c00-istiod-ca-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.662243 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.662044    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9vdg\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-kube-api-access-d9vdg\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.662243 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.662059    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-token\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.662243 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.662072    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0edab5d4-d7f4-401f-8176-2f624de66c00-istio-envoy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:03.830637 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.830555    2558 generic.go:356] "Generic (PLEG): container finished" podID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerID="ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e" exitCode=0
Sep 10 15:05:03.830637 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.830614    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"
Sep 10 15:05:03.830825 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.830637    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" event={"ID":"0edab5d4-d7f4-401f-8176-2f624de66c00","Type":"ContainerDied","Data":"ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e"}
Sep 10 15:05:03.830825 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.830683    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2" event={"ID":"0edab5d4-d7f4-401f-8176-2f624de66c00","Type":"ContainerDied","Data":"8274d1a41a51ce8748dec9a703c71fa811e0458e01938b636a67f1ad82a80b4d"}
Sep 10 15:05:03.830825 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.830703    2558 scope.go:117] "RemoveContainer" containerID="ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e"
Sep 10 15:05:03.841588 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.841568    2558 scope.go:117] "RemoveContainer" containerID="2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54"
Sep 10 15:05:03.852466 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.852445    2558 scope.go:117] "RemoveContainer" containerID="ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e"
Sep 10 15:05:03.852785 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:05:03.852761    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e\": container with ID starting with ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e not found: ID does not exist" containerID="ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e"
Sep 10 15:05:03.852867 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.852795    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e"} err="failed to get container status \"ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e\": rpc error: code = NotFound desc = could not find container \"ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e\": container with ID starting with ab4212e07921b73e9f7e959a941d94e26147860fc173682f208627fcd88c5f8e not found: ID does not exist"
Sep 10 15:05:03.852867 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.852812    2558 scope.go:117] "RemoveContainer" containerID="2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54"
Sep 10 15:05:03.853080 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:05:03.853065    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54\": container with ID starting with 2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54 not found: ID does not exist" containerID="2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54"
Sep 10 15:05:03.853135 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.853082    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54"} err="failed to get container status \"2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54\": rpc error: code = NotFound desc = could not find container \"2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54\": container with ID starting with 2816c2fecda72ee506b55f67e63700285e3394b694b1d5b8f1296c95732b4b54 not found: ID does not exist"
Sep 10 15:05:03.854811 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.854784    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"]
Sep 10 15:05:03.857577 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:03.857554    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-cb65dc-openshift-default-6d6fd7cdb8-872s2"]
Sep 10 15:05:04.018923 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:04.018888    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" path="/var/lib/kubelet/pods/0edab5d4-d7f4-401f-8176-2f624de66c00/volumes"
Sep 10 15:05:09.229691 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:09.229657    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"]
Sep 10 15:05:09.233173 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:09.233149    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-5fb994-29817540-24mv6"]
Sep 10 15:05:10.019171 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:10.019136    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fd61dbf-093e-4824-bc80-46129214542f" path="/var/lib/kubelet/pods/8fd61dbf-093e-4824-bc80-46129214542f/volumes"
Sep 10 15:05:12.936609 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:12.936571    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st" podUID="7e5f8bf1-b7f0-43f9-93b1-29ed57127215" containerName="main" containerID="cri-o://43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6" gracePeriod=45
Sep 10 15:05:13.190187 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.190164    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:05:13.256639 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.256612    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tls-certs\") pod \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") "
Sep 10 15:05:13.256823 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.256673    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7wjm\" (UniqueName: \"kubernetes.io/projected/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-kube-api-access-m7wjm\") pod \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") "
Sep 10 15:05:13.256823 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.256701    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-dshm\") pod \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") "
Sep 10 15:05:13.256823 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.256756    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-model-cache\") pod \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") "
Sep 10 15:05:13.256823 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.256793    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tmp-dir\") pod \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") "
Sep 10 15:05:13.256823 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.256808    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-home\") pod \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\" (UID: \"7e5f8bf1-b7f0-43f9-93b1-29ed57127215\") "
Sep 10 15:05:13.257106 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.257068    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e5f8bf1-b7f0-43f9-93b1-29ed57127215" (UID: "7e5f8bf1-b7f0-43f9-93b1-29ed57127215"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:13.257106 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.257087    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e5f8bf1-b7f0-43f9-93b1-29ed57127215" (UID: "7e5f8bf1-b7f0-43f9-93b1-29ed57127215"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:13.257255 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.257183    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-home" (OuterVolumeSpecName: "home") pod "7e5f8bf1-b7f0-43f9-93b1-29ed57127215" (UID: "7e5f8bf1-b7f0-43f9-93b1-29ed57127215"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:13.259028 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.258996    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-dshm" (OuterVolumeSpecName: "dshm") pod "7e5f8bf1-b7f0-43f9-93b1-29ed57127215" (UID: "7e5f8bf1-b7f0-43f9-93b1-29ed57127215"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:13.259028 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.259020    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-kube-api-access-m7wjm" (OuterVolumeSpecName: "kube-api-access-m7wjm") pod "7e5f8bf1-b7f0-43f9-93b1-29ed57127215" (UID: "7e5f8bf1-b7f0-43f9-93b1-29ed57127215"). InnerVolumeSpecName "kube-api-access-m7wjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:05:13.259177 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.259041    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e5f8bf1-b7f0-43f9-93b1-29ed57127215" (UID: "7e5f8bf1-b7f0-43f9-93b1-29ed57127215"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:05:13.358270 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.358210    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7wjm\" (UniqueName: \"kubernetes.io/projected/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-kube-api-access-m7wjm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:13.358270 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.358264    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:13.358270 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.358276    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:13.358514 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.358302    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:13.358514 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.358310    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:13.358514 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.358317    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e5f8bf1-b7f0-43f9-93b1-29ed57127215-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:13.895150 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.895114    2558 generic.go:356] "Generic (PLEG): container finished" podID="7e5f8bf1-b7f0-43f9-93b1-29ed57127215" containerID="43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6" exitCode=0
Sep 10 15:05:13.895352 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.895163    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"
Sep 10 15:05:13.895352 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.895202    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st" event={"ID":"7e5f8bf1-b7f0-43f9-93b1-29ed57127215","Type":"ContainerDied","Data":"43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6"}
Sep 10 15:05:13.895352 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.895275    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st" event={"ID":"7e5f8bf1-b7f0-43f9-93b1-29ed57127215","Type":"ContainerDied","Data":"64ce1db7cfc83b5256812044668d85fbcaa6d210f438a531b5a9c2a3b1e8350e"}
Sep 10 15:05:13.895352 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.895296    2558 scope.go:117] "RemoveContainer" containerID="43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6"
Sep 10 15:05:13.905625 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.905609    2558 scope.go:117] "RemoveContainer" containerID="43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6"
Sep 10 15:05:13.905881 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:05:13.905864    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6\": container with ID starting with 43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6 not found: ID does not exist" containerID="43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6"
Sep 10 15:05:13.905922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.905890    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6"} err="failed to get container status \"43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6\": rpc error: code = NotFound desc = could not find container \"43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6\": container with ID starting with 43380a453195fd6926de961ec2da1dfbe2987f0475072f99b178061a9e4f41f6 not found: ID does not exist"
Sep 10 15:05:13.917990 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.917963    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"]
Sep 10 15:05:13.922391 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:13.922366    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/sub-test-model-16ad45-kserve-779d6cd9cc-9k9st"]
Sep 10 15:05:14.019159 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.019124    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e5f8bf1-b7f0-43f9-93b1-29ed57127215" path="/var/lib/kubelet/pods/7e5f8bf1-b7f0-43f9-93b1-29ed57127215/volumes"
Sep 10 15:05:14.186696 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.186613    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4" podUID="df5840df-93c3-420b-903a-425f04aca384" containerName="main" containerID="cri-o://e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd" gracePeriod=45
Sep 10 15:05:14.442356 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.442292    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:05:14.471006 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.470976    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5840df-93c3-420b-903a-425f04aca384-tls-certs\") pod \"df5840df-93c3-420b-903a-425f04aca384\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") "
Sep 10 15:05:14.471186 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471038    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-home\") pod \"df5840df-93c3-420b-903a-425f04aca384\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") "
Sep 10 15:05:14.471186 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471064    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gnwj\" (UniqueName: \"kubernetes.io/projected/df5840df-93c3-420b-903a-425f04aca384-kube-api-access-9gnwj\") pod \"df5840df-93c3-420b-903a-425f04aca384\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") "
Sep 10 15:05:14.471186 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471096    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-tmp-dir\") pod \"df5840df-93c3-420b-903a-425f04aca384\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") "
Sep 10 15:05:14.471186 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471140    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-model-cache\") pod \"df5840df-93c3-420b-903a-425f04aca384\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") "
Sep 10 15:05:14.471186 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471184    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-dshm\") pod \"df5840df-93c3-420b-903a-425f04aca384\" (UID: \"df5840df-93c3-420b-903a-425f04aca384\") "
Sep 10 15:05:14.471572 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471344    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-home" (OuterVolumeSpecName: "home") pod "df5840df-93c3-420b-903a-425f04aca384" (UID: "df5840df-93c3-420b-903a-425f04aca384"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:14.471572 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471397    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df5840df-93c3-420b-903a-425f04aca384" (UID: "df5840df-93c3-420b-903a-425f04aca384"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:14.471572 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471445    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-model-cache" (OuterVolumeSpecName: "model-cache") pod "df5840df-93c3-420b-903a-425f04aca384" (UID: "df5840df-93c3-420b-903a-425f04aca384"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:14.471738 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471592    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:14.471738 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471611    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:14.471738 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.471626    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:14.473375 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.473347    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-dshm" (OuterVolumeSpecName: "dshm") pod "df5840df-93c3-420b-903a-425f04aca384" (UID: "df5840df-93c3-420b-903a-425f04aca384"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:14.473488 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.473402    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df5840df-93c3-420b-903a-425f04aca384-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df5840df-93c3-420b-903a-425f04aca384" (UID: "df5840df-93c3-420b-903a-425f04aca384"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:05:14.473488 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.473459    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df5840df-93c3-420b-903a-425f04aca384-kube-api-access-9gnwj" (OuterVolumeSpecName: "kube-api-access-9gnwj") pod "df5840df-93c3-420b-903a-425f04aca384" (UID: "df5840df-93c3-420b-903a-425f04aca384"). InnerVolumeSpecName "kube-api-access-9gnwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:05:14.572645 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.572608    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df5840df-93c3-420b-903a-425f04aca384-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:14.572645 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.572640    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df5840df-93c3-420b-903a-425f04aca384-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:14.572645 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.572651    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gnwj\" (UniqueName: \"kubernetes.io/projected/df5840df-93c3-420b-903a-425f04aca384-kube-api-access-9gnwj\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:14.904026 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.903935    2558 generic.go:356] "Generic (PLEG): container finished" podID="df5840df-93c3-420b-903a-425f04aca384" containerID="e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd" exitCode=0
Sep 10 15:05:14.904026 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.903979    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"
Sep 10 15:05:14.904238 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.904026    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4" event={"ID":"df5840df-93c3-420b-903a-425f04aca384","Type":"ContainerDied","Data":"e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd"}
Sep 10 15:05:14.904238 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.904064    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4" event={"ID":"df5840df-93c3-420b-903a-425f04aca384","Type":"ContainerDied","Data":"dd4a22afee7af64034a3752c7926ad87e83155616aaa7f8964aa3fdc20bcba07"}
Sep 10 15:05:14.904238 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.904086    2558 scope.go:117] "RemoveContainer" containerID="e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd"
Sep 10 15:05:14.914212 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.914192    2558 scope.go:117] "RemoveContainer" containerID="e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd"
Sep 10 15:05:14.914579 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:05:14.914561    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd\": container with ID starting with e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd not found: ID does not exist" containerID="e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd"
Sep 10 15:05:14.914630 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.914588    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd"} err="failed to get container status \"e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd\": rpc error: code = NotFound desc = could not find container \"e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd\": container with ID starting with e50d4a068b0e2e0638a0c31e844a5d5866e54d12067c48c8bd408343408ed8fd not found: ID does not exist"
Sep 10 15:05:14.926280 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.926250    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"]
Sep 10 15:05:14.930904 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:14.930878    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-3f81b7/sub-test-model-3f81b7-kserve-5bf9784bdf-wh2h4"]
Sep 10 15:05:15.850700 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:15.850670    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"]
Sep 10 15:05:15.851179 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:15.850955    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy" containerID="cri-o://f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115" gracePeriod=30
Sep 10 15:05:16.019414 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.019382    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df5840df-93c3-420b-903a-425f04aca384" path="/var/lib/kubelet/pods/df5840df-93c3-420b-903a-425f04aca384/volumes"
Sep 10 15:05:16.208911 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.208879    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"]
Sep 10 15:05:16.209299 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209286    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df5840df-93c3-420b-903a-425f04aca384" containerName="main"
Sep 10 15:05:16.209349 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209301    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="df5840df-93c3-420b-903a-425f04aca384" containerName="main"
Sep 10 15:05:16.209349 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209311    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82940ddd-71f3-4127-9a00-473533b16bba" containerName="test-curl-7659-74cf8a"
Sep 10 15:05:16.209349 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209317    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="82940ddd-71f3-4127-9a00-473533b16bba" containerName="test-curl-7659-74cf8a"
Sep 10 15:05:16.209349 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209324    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy"
Sep 10 15:05:16.209349 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209329    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy"
Sep 10 15:05:16.209349 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209343    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy"
Sep 10 15:05:16.209349 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209349    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy"
Sep 10 15:05:16.209553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209360    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e5f8bf1-b7f0-43f9-93b1-29ed57127215" containerName="main"
Sep 10 15:05:16.209553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209365    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e5f8bf1-b7f0-43f9-93b1-29ed57127215" containerName="main"
Sep 10 15:05:16.209553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209441    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e5f8bf1-b7f0-43f9-93b1-29ed57127215" containerName="main"
Sep 10 15:05:16.209553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209448    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy"
Sep 10 15:05:16.209553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209456    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="df5840df-93c3-420b-903a-425f04aca384" containerName="main"
Sep 10 15:05:16.209553 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.209462    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="82940ddd-71f3-4127-9a00-473533b16bba" containerName="test-curl-7659-74cf8a"
Sep 10 15:05:16.213609 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.213590    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"
Sep 10 15:05:16.216570 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.216547    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:05:16.216740 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.216680    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:05:16.220514 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.220481    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"]
Sep 10 15:05:16.291637 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.291588    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf72379-6813-4dfe-9bda-5c4146304b67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w\" (UID: \"8bf72379-6813-4dfe-9bda-5c4146304b67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"
Sep 10 15:05:16.392581 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.392549    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf72379-6813-4dfe-9bda-5c4146304b67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w\" (UID: \"8bf72379-6813-4dfe-9bda-5c4146304b67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"
Sep 10 15:05:16.393173 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.393154    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf72379-6813-4dfe-9bda-5c4146304b67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w\" (UID: \"8bf72379-6813-4dfe-9bda-5c4146304b67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"
Sep 10 15:05:16.526287 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.526192    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"
Sep 10 15:05:16.799584 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.799553    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"]
Sep 10 15:05:16.800683 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:05:16.800657    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bf72379_6813_4dfe_9bda_5c4146304b67.slice/crio-9780d9aa0857af5967dc7db9590e8d7a1f8a515f70c9bed797c73bd065753d07 WatchSource:0}: Error finding container 9780d9aa0857af5967dc7db9590e8d7a1f8a515f70c9bed797c73bd065753d07: Status 404 returned error can't find the container with id 9780d9aa0857af5967dc7db9590e8d7a1f8a515f70c9bed797c73bd065753d07
Sep 10 15:05:16.918666 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:16.918628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w" event={"ID":"8bf72379-6813-4dfe-9bda-5c4146304b67","Type":"ContainerStarted","Data":"9780d9aa0857af5967dc7db9590e8d7a1f8a515f70c9bed797c73bd065753d07"}
Sep 10 15:05:17.930703 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:17.930619    2558 generic.go:356] "Generic (PLEG): container finished" podID="8bf72379-6813-4dfe-9bda-5c4146304b67" containerID="585f9521bd15642d3b45f7f69943d46cfaf67ba3c4b20027bf4e3ac2515bee7f" exitCode=0
Sep 10 15:05:17.930703 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:17.930661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w" event={"ID":"8bf72379-6813-4dfe-9bda-5c4146304b67","Type":"ContainerDied","Data":"585f9521bd15642d3b45f7f69943d46cfaf67ba3c4b20027bf4e3ac2515bee7f"}
Sep 10 15:05:19.070982 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.070954    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"
Sep 10 15:05:19.121744 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.121709    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf72379-6813-4dfe-9bda-5c4146304b67-maas-api-service-ca\") pod \"8bf72379-6813-4dfe-9bda-5c4146304b67\" (UID: \"8bf72379-6813-4dfe-9bda-5c4146304b67\") "
Sep 10 15:05:19.122083 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.122051    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bf72379-6813-4dfe-9bda-5c4146304b67-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8bf72379-6813-4dfe-9bda-5c4146304b67" (UID: "8bf72379-6813-4dfe-9bda-5c4146304b67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:05:19.122271 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.122250    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bf72379-6813-4dfe-9bda-5c4146304b67-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:19.177451 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.177417    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:05:19.944238 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.944188    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"
Sep 10 15:05:19.944415 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.944243    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w" event={"ID":"8bf72379-6813-4dfe-9bda-5c4146304b67","Type":"ContainerDied","Data":"9780d9aa0857af5967dc7db9590e8d7a1f8a515f70c9bed797c73bd065753d07"}
Sep 10 15:05:19.944415 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:19.944291    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9780d9aa0857af5967dc7db9590e8d7a1f8a515f70c9bed797c73bd065753d07"
Sep 10 15:05:21.105121 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.105094    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 15:05:21.242753 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.242722    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-data\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.242948 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.242776    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-socket\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.242948 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.242795    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-istio-token\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.242948 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.242810    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2dd132-fabe-4915-a362-acc938885e52-istio-podinfo\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.242948 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.242936    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2dd132-fabe-4915-a362-acc938885e52-istiod-ca-cert\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.243191 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243022    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-envoy\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.243191 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243025    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:21.243191 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243040    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:21.243191 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243059    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-credential-socket\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.243191 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243089    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blh5n\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-kube-api-access-blh5n\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.243191 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243125    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-certs\") pod \"9c2dd132-fabe-4915-a362-acc938885e52\" (UID: \"9c2dd132-fabe-4915-a362-acc938885e52\") "
Sep 10 15:05:21.243559 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243351    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:21.243559 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243378    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c2dd132-fabe-4915-a362-acc938885e52-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:05:21.243559 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243522    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:21.243895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243695    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-credential-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.243895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243762    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-data\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.243895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243776    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.243895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.243790    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c2dd132-fabe-4915-a362-acc938885e52-istiod-ca-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.245387 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.245270    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:05:21.245387 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.245359    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-kube-api-access-blh5n" (OuterVolumeSpecName: "kube-api-access-blh5n") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "kube-api-access-blh5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:05:21.245387 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.245361    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c2dd132-fabe-4915-a362-acc938885e52-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 15:05:21.245559 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.245388    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c2dd132-fabe-4915-a362-acc938885e52" (UID: "9c2dd132-fabe-4915-a362-acc938885e52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:05:21.344860 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.344824    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-istio-envoy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.344860 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.344852    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blh5n\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-kube-api-access-blh5n\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.344860 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.344862    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c2dd132-fabe-4915-a362-acc938885e52-workload-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.344860 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.344869    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c2dd132-fabe-4915-a362-acc938885e52-istio-token\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.345114 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.344877    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c2dd132-fabe-4915-a362-acc938885e52-istio-podinfo\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:05:21.960658 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.960622    2558 generic.go:356] "Generic (PLEG): container finished" podID="9c2dd132-fabe-4915-a362-acc938885e52" containerID="f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115" exitCode=0
Sep 10 15:05:21.960833 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.960670    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"
Sep 10 15:05:21.960833 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.960709    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" event={"ID":"9c2dd132-fabe-4915-a362-acc938885e52","Type":"ContainerDied","Data":"f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115"}
Sep 10 15:05:21.960833 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.960763    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q" event={"ID":"9c2dd132-fabe-4915-a362-acc938885e52","Type":"ContainerDied","Data":"7ef1d699a1e13c3a795d73380d71b2abb887cac72e2c11ba7129a1ca0bf3c76c"}
Sep 10 15:05:21.960833 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.960793    2558 scope.go:117] "RemoveContainer" containerID="f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115"
Sep 10 15:05:21.971153 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.971002    2558 scope.go:117] "RemoveContainer" containerID="b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617"
Sep 10 15:05:21.985185 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.985078    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"]
Sep 10 15:05:21.985301 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.985258    2558 scope.go:117] "RemoveContainer" containerID="f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115"
Sep 10 15:05:21.985613 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:05:21.985596    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115\": container with ID starting with f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115 not found: ID does not exist" containerID="f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115"
Sep 10 15:05:21.985659 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.985621    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115"} err="failed to get container status \"f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115\": rpc error: code = NotFound desc = could not find container \"f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115\": container with ID starting with f9cbd4152a2673a4a641994f3dbb91829ae8faa26458da9a21e5d49ef52da115 not found: ID does not exist"
Sep 10 15:05:21.985659 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.985636    2558 scope.go:117] "RemoveContainer" containerID="b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617"
Sep 10 15:05:21.985901 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:05:21.985883    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617\": container with ID starting with b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617 not found: ID does not exist" containerID="b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617"
Sep 10 15:05:21.985941 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.985906    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617"} err="failed to get container status \"b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617\": rpc error: code = NotFound desc = could not find container \"b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617\": container with ID starting with b76edbfbb9cde1be50ddd8546b061ce6d6d1c6e4afa28f2bacffaa7144279617 not found: ID does not exist"
Sep 10 15:05:21.991947 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:21.991923    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-5fb994-openshift-default-9648ff944-qvb9q"]
Sep 10 15:05:22.019849 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:22.019817    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c2dd132-fabe-4915-a362-acc938885e52" path="/var/lib/kubelet/pods/9c2dd132-fabe-4915-a362-acc938885e52/volumes"
Sep 10 15:05:23.874431 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874401    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"]
Sep 10 15:05:23.874811 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874784    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf72379-6813-4dfe-9bda-5c4146304b67" containerName="revoke-keys"
Sep 10 15:05:23.874811 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874795    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf72379-6813-4dfe-9bda-5c4146304b67" containerName="revoke-keys"
Sep 10 15:05:23.874935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874839    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy"
Sep 10 15:05:23.874935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874844    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy"
Sep 10 15:05:23.874935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874852    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy"
Sep 10 15:05:23.874935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874857    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy"
Sep 10 15:05:23.874935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874914    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy"
Sep 10 15:05:23.874935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874921    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0edab5d4-d7f4-401f-8176-2f624de66c00" containerName="istio-proxy"
Sep 10 15:05:23.874935 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.874932    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf72379-6813-4dfe-9bda-5c4146304b67" containerName="revoke-keys"
Sep 10 15:05:23.879207 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.879192    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:23.881858 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.881825    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"openshift-service-ca.crt\""
Sep 10 15:05:23.881982 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.881864    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"default-dockercfg-xs8tp\""
Sep 10 15:05:23.883320 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.883297    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"rate-test-model-16ad45-kserve-self-signed-certs\""
Sep 10 15:05:23.883320 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.883315    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-16ad45\"/\"kube-root-ca.crt\""
Sep 10 15:05:23.887317 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.887289    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"]
Sep 10 15:05:23.972453 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.972420    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-home\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:23.972453 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.972458    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tls-certs\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:23.972758 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.972500    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tmp-dir\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:23.972758 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.972609    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpkvz\" (UniqueName: \"kubernetes.io/projected/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-kube-api-access-kpkvz\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:23.972758 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.972680    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-dshm\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:23.972758 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:23.972724    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-model-cache\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073283 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073210    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tmp-dir\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073487 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpkvz\" (UniqueName: \"kubernetes.io/projected/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-kube-api-access-kpkvz\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073487 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073375    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-dshm\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073487 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073402    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-model-cache\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073487 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073444    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-home\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073487 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073481    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tls-certs\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073752 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tmp-dir\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073810 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073790    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-model-cache\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.073864 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.073839    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-home\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.075764 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.075745    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-dshm\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.076041 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.076023    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tls-certs\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.082543 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.082518    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpkvz\" (UniqueName: \"kubernetes.io/projected/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-kube-api-access-kpkvz\") pod \"rate-test-model-16ad45-kserve-fb756d875-bd94t\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") " pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.189926 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.189896    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:24.470800 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.470767    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"]
Sep 10 15:05:24.471895 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:05:24.471864    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded27428a_bade_4837_bf2d_a9aa5e5b1cbb.slice/crio-d5d3c7bda28fb079b1d26b23079a50837892b8403a3e934c213e9c6a979c19f1 WatchSource:0}: Error finding container d5d3c7bda28fb079b1d26b23079a50837892b8403a3e934c213e9c6a979c19f1: Status 404 returned error can't find the container with id d5d3c7bda28fb079b1d26b23079a50837892b8403a3e934c213e9c6a979c19f1
Sep 10 15:05:24.984505 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.984471    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t" event={"ID":"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb","Type":"ContainerStarted","Data":"82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d"}
Sep 10 15:05:24.984505 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:24.984512    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t" event={"ID":"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb","Type":"ContainerStarted","Data":"d5d3c7bda28fb079b1d26b23079a50837892b8403a3e934c213e9c6a979c19f1"}
Sep 10 15:05:25.006345 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:25.006277    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t" podStartSLOduration=2.006260684 podStartE2EDuration="2.006260684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:05:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:05:25.004113361 +0000 UTC m=+1369.490801323" watchObservedRunningTime="2026-09-10 15:05:25.006260684 +0000 UTC m=+1369.492948576"
Sep 10 15:05:26.890523 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:26.890478    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"]
Sep 10 15:05:26.893051 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:26.893025    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-5899e8-29817540-vrv4x"]
Sep 10 15:05:27.115715 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:27.115674    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"]
Sep 10 15:05:27.119034 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:27.119007    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5899e8-bvs4w"]
Sep 10 15:05:28.018295 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:28.018249    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf72379-6813-4dfe-9bda-5c4146304b67" path="/var/lib/kubelet/pods/8bf72379-6813-4dfe-9bda-5c4146304b67/volumes"
Sep 10 15:05:28.018847 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:28.018586    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aea05a25-bdad-4802-8908-8a692a320774" path="/var/lib/kubelet/pods/aea05a25-bdad-4802-8908-8a692a320774/volumes"
Sep 10 15:05:29.878097 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:29.878055    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:05:34.190834 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:34.190789    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:34.191276 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:34.190872    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:34.203606 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:34.203578    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:35.055535 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:35.055509    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:05:36.159478 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:36.159449    2558 scope.go:117] "RemoveContainer" containerID="a3f8318442413c1b316256643fd5c573ca86ec59d1d07a8f8c47aad3b696a793"
Sep 10 15:05:36.171302 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:05:36.171278    2558 scope.go:117] "RemoveContainer" containerID="1b00d991271bbac9bbe05b4c3ef2849653396d1e3e6810b5140cdf1321546d77"
Sep 10 15:06:07.176973 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:07.176935    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:06:22.078922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:22.078883    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:06:31.669955 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:31.669842    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:06:36.227546 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:36.227511    2558 scope.go:117] "RemoveContainer" containerID="519682c8051ea97f2651177602acd18b706931519d75a78f42a1afe817491143"
Sep 10 15:06:36.239698 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:36.239675    2558 scope.go:117] "RemoveContainer" containerID="496e032da7a0f0b729f614174a6bdf548e8f8337f0a96e1ada50a96d36be4a8f"
Sep 10 15:06:36.248478 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:36.248455    2558 scope.go:117] "RemoveContainer" containerID="2b5383035093ca2f474fa380002e89f3c9aaccc849b38c2e459ab787b06a7dc6"
Sep 10 15:06:38.220430 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:38.220396    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"]
Sep 10 15:06:42.385310 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:42.385273    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:06:53.241258 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.241195    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t" podUID="ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" containerName="main" containerID="cri-o://82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d" gracePeriod=45
Sep 10 15:06:53.487096 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.487074    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:06:53.523397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.523312    2558 generic.go:356] "Generic (PLEG): container finished" podID="ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" containerID="82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d" exitCode=0
Sep 10 15:06:53.523397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.523358    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"
Sep 10 15:06:53.523641 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.523398    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t" event={"ID":"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb","Type":"ContainerDied","Data":"82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d"}
Sep 10 15:06:53.523641 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.523451    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t" event={"ID":"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb","Type":"ContainerDied","Data":"d5d3c7bda28fb079b1d26b23079a50837892b8403a3e934c213e9c6a979c19f1"}
Sep 10 15:06:53.523641 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.523473    2558 scope.go:117] "RemoveContainer" containerID="82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d"
Sep 10 15:06:53.532498 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.532479    2558 scope.go:117] "RemoveContainer" containerID="82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d"
Sep 10 15:06:53.532804 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:06:53.532778    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d\": container with ID starting with 82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d not found: ID does not exist" containerID="82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d"
Sep 10 15:06:53.532859 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.532818    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d"} err="failed to get container status \"82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d\": rpc error: code = NotFound desc = could not find container \"82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d\": container with ID starting with 82fc9ecf6277a72711052e2433a75ba5b8e2236c200c268cb018489c212f1b4d not found: ID does not exist"
Sep 10 15:06:53.639508 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639473    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-dshm\") pod \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") "
Sep 10 15:06:53.639669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639559    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tmp-dir\") pod \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") "
Sep 10 15:06:53.639669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639591    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-home\") pod \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") "
Sep 10 15:06:53.639669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639607    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpkvz\" (UniqueName: \"kubernetes.io/projected/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-kube-api-access-kpkvz\") pod \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") "
Sep 10 15:06:53.639669 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639624    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-model-cache\") pod \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") "
Sep 10 15:06:53.639974 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639705    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tls-certs\") pod \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\" (UID: \"ed27428a-bade-4837-bf2d-a9aa5e5b1cbb\") "
Sep 10 15:06:53.639974 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639866    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-home" (OuterVolumeSpecName: "home") pod "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" (UID: "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:06:53.639974 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639878    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" (UID: "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:06:53.639974 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.639955    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" (UID: "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:06:53.640156 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.640046    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:53.640156 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.640066    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:53.640156 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.640079    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:53.641819 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.641798    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-kube-api-access-kpkvz" (OuterVolumeSpecName: "kube-api-access-kpkvz") pod "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" (UID: "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb"). InnerVolumeSpecName "kube-api-access-kpkvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:06:53.641895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.641836    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-dshm" (OuterVolumeSpecName: "dshm") pod "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" (UID: "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:06:53.641895 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.641869    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" (UID: "ed27428a-bade-4837-bf2d-a9aa5e5b1cbb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:06:53.741541 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.741497    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:53.741541 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.741535    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:53.741541 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.741545    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpkvz\" (UniqueName: \"kubernetes.io/projected/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb-kube-api-access-kpkvz\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:06:53.848499 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.848466    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"]
Sep 10 15:06:53.852289 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:53.852265    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-16ad45/rate-test-model-16ad45-kserve-fb756d875-bd94t"]
Sep 10 15:06:54.019084 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:54.019053    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" path="/var/lib/kubelet/pods/ed27428a-bade-4837-bf2d-a9aa5e5b1cbb/volumes"
Sep 10 15:06:55.479066 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:06:55.479033    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:07:09.362055 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.362018    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"]
Sep 10 15:07:09.362567 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.362493    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" containerName="main"
Sep 10 15:07:09.362567 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.362511    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" containerName="main"
Sep 10 15:07:09.362754 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.362598    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c2dd132-fabe-4915-a362-acc938885e52" containerName="istio-proxy"
Sep 10 15:07:09.362754 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.362607    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed27428a-bade-4837-bf2d-a9aa5e5b1cbb" containerName="main"
Sep 10 15:07:09.365637 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.365619    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.368465 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.368444    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-cb5c8c-openshift-default-dockercfg-t9bd7\""
Sep 10 15:07:09.382028 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.382005    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"]
Sep 10 15:07:09.479679 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.479643    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-envoy\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.479874 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.479688    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-token\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.479874 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.479787    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-socket\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.479874 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.479833    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-podinfo\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.479995 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.479913    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-certs\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.479995 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.479965    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-data\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.479995 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.479984    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12fc793b-fb84-4d61-8128-e0d9826d6a14-istiod-ca-cert\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.480109 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.480005    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcfjr\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-kube-api-access-tcfjr\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.480109 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.480044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-credential-socket\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.581558 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.581526    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-envoy\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.581558 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.581567    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-token\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.581809 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.581702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-socket\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.581809 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.581747    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-podinfo\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.581946 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.581886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-certs\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582012 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.581964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-data\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582012 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.581997    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12fc793b-fb84-4d61-8128-e0d9826d6a14-istiod-ca-cert\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582121 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.582019    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcfjr\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-kube-api-access-tcfjr\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582121 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.582077    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-credential-socket\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582121 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.582108    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-socket\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582329 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.582162    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-certs\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582396 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.582370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-data\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.582459 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.582398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-credential-socket\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.583001 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.582978    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12fc793b-fb84-4d61-8128-e0d9826d6a14-istiod-ca-cert\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.583970 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.583947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-envoy\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.584426 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.584407    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-podinfo\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.593569 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.593540    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcfjr\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-kube-api-access-tcfjr\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.593741 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.593722    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-token\") pod \"e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") " pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.675987 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.675902    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:09.955734 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.955701    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"]
Sep 10 15:07:09.957196 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:07:09.957150    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fc793b_fb84_4d61_8128_e0d9826d6a14.slice/crio-a3126df29e79db2899442b65cdf6696da79eed230f9271061274738c270d6c8e WatchSource:0}: Error finding container a3126df29e79db2899442b65cdf6696da79eed230f9271061274738c270d6c8e: Status 404 returned error can't find the container with id a3126df29e79db2899442b65cdf6696da79eed230f9271061274738c270d6c8e
Sep 10 15:07:09.959444 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.959407    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:07:09.959538 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.959478    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:07:09.959538 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:09.959506    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:07:10.634075 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:10.634034    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" event={"ID":"12fc793b-fb84-4d61-8128-e0d9826d6a14","Type":"ContainerStarted","Data":"e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30"}
Sep 10 15:07:10.634499 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:10.634091    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" event={"ID":"12fc793b-fb84-4d61-8128-e0d9826d6a14","Type":"ContainerStarted","Data":"a3126df29e79db2899442b65cdf6696da79eed230f9271061274738c270d6c8e"}
Sep 10 15:07:10.659512 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:10.659456    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" podStartSLOduration=1.6594363460000001 podStartE2EDuration="1.659436346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:07:10.659083399 +0000 UTC m=+1475.145771291" watchObservedRunningTime="2026-09-10 15:07:10.659436346 +0000 UTC m=+1475.146124239"
Sep 10 15:07:10.676407 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:10.676376    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:11.682189 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:11.682156    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:12.648566 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:12.648536    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:12.649551 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:12.649533    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:20.885608 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:20.885548    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:07:30.761603 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:30.761573    2558 generic.go:356] "Generic (PLEG): container finished" podID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerID="e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30" exitCode=0
Sep 10 15:07:30.762002 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:30.761643    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" event={"ID":"12fc793b-fb84-4d61-8128-e0d9826d6a14","Type":"ContainerDied","Data":"e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30"}
Sep 10 15:07:30.762098 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:30.762073    2558 scope.go:117] "RemoveContainer" containerID="e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30"
Sep 10 15:07:30.763292 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:30.763264    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:07:30.763372 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:30.763336    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:07:30.763372 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:30.763369    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892136Ki","pods":"250"}
Sep 10 15:07:31.768994 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:31.768953    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" event={"ID":"12fc793b-fb84-4d61-8128-e0d9826d6a14","Type":"ContainerStarted","Data":"95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212"}
Sep 10 15:07:32.676297 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:32.676257    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:32.676574 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:32.676552    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 15:07:32.676626 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:32.676605    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 15:07:33.676558 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:33.676517    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 15:07:33.677127 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:33.676576    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 15:07:34.676850 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:34.676813    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 15:07:34.677281 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:34.676878    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 15:07:35.677324 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:35.677278    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 15:07:35.677776 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:35.677353    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 15:07:36.278940 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:36.278894    2558 scope.go:117] "RemoveContainer" containerID="37bfb02f9670cac57fad41acb0adde4154e67c494205133ca373fa22b976e1c0"
Sep 10 15:07:36.676603 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:36.676514    2558 patch_prober.go:28] interesting pod/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 15:07:36.676603 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:36.676578    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 15:07:37.679990 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:37.679960    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:37.680385 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:37.680252    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:37.681051 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:37.681032    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:07:54.093696 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.093662    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"]
Sep 10 15:07:54.098554 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.098529    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.101289 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.101264    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b022ee\"/\"default-dockercfg-rrdlm\""
Sep 10 15:07:54.101451 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.101310    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b022ee\"/\"kube-root-ca.crt\""
Sep 10 15:07:54.102278 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.102259    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-b022ee\"/\"ipp-route-b022ee-kserve-self-signed-certs\""
Sep 10 15:07:54.102405 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.102292    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-b022ee\"/\"openshift-service-ca.crt\""
Sep 10 15:07:54.109119 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.109100    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"]
Sep 10 15:07:54.232745 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.232697    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-model-cache\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.232949 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.232756    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9447\" (UniqueName: \"kubernetes.io/projected/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-kube-api-access-w9447\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.232949 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.232815    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-home\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.232949 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.232856    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-dshm\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.232949 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.232896    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tls-certs\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.232949 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.232935    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tmp-dir\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334461 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334423    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-model-cache\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334635 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334473    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9447\" (UniqueName: \"kubernetes.io/projected/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-kube-api-access-w9447\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334635 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334525    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-home\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334635 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334548    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-dshm\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334635 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334567    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tls-certs\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334635 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334607    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tmp-dir\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-model-cache\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.334932 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334893    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-home\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.335040 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.334967    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tmp-dir\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.336942 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.336923    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-dshm\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.337258 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.337213    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tls-certs\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.343661 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.343638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9447\" (UniqueName: \"kubernetes.io/projected/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-kube-api-access-w9447\") pod \"ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") " pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.409106 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.409016    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:07:54.684000 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.683969    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"]
Sep 10 15:07:54.684722 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:07:54.684698    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fe53ddb_8fdf_4d8e_a553_9a614adc9ce3.slice/crio-9f34e90f4afe515489faded254e1d6f1a7014d047ab050cc72c3f26861c00c01 WatchSource:0}: Error finding container 9f34e90f4afe515489faded254e1d6f1a7014d047ab050cc72c3f26861c00c01: Status 404 returned error can't find the container with id 9f34e90f4afe515489faded254e1d6f1a7014d047ab050cc72c3f26861c00c01
Sep 10 15:07:54.917508 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.917471    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6" event={"ID":"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3","Type":"ContainerStarted","Data":"b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0"}
Sep 10 15:07:54.917508 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.917515    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6" event={"ID":"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3","Type":"ContainerStarted","Data":"9f34e90f4afe515489faded254e1d6f1a7014d047ab050cc72c3f26861c00c01"}
Sep 10 15:07:54.934760 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:54.934645    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6" podStartSLOduration=0.934629192 podStartE2EDuration="934.629192ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:07:54.933573387 +0000 UTC m=+1519.420261306" watchObservedRunningTime="2026-09-10 15:07:54.934629192 +0000 UTC m=+1519.421317083"
Sep 10 15:07:58.280528 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:07:58.280446    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:08:04.409283 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:08:04.409250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:08:04.409654 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:08:04.409298    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:08:04.421984 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:08:04.421959    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:08:04.987279 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:08:04.987246    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:08:42.991665 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:08:42.991624    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:09:02.926268 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:02.926229    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:09:18.809983 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:18.809950    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"]
Sep 10 15:09:18.813339 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:18.813322    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"
Sep 10 15:09:18.816067 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:18.816046    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:09:18.816165 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:18.816139    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:09:18.822279 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:18.821984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"]
Sep 10 15:09:18.984959 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:18.984913    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf2e8b-b4f7-412a-a290-228381c5e89f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8\" (UID: \"c6bf2e8b-b4f7-412a-a290-228381c5e89f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"
Sep 10 15:09:19.085632 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:19.085537    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf2e8b-b4f7-412a-a290-228381c5e89f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8\" (UID: \"c6bf2e8b-b4f7-412a-a290-228381c5e89f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"
Sep 10 15:09:19.086410 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:19.086383    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf2e8b-b4f7-412a-a290-228381c5e89f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8\" (UID: \"c6bf2e8b-b4f7-412a-a290-228381c5e89f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"
Sep 10 15:09:19.125452 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:19.125404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"
Sep 10 15:09:19.402280 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:19.402255    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"]
Sep 10 15:09:19.404241 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:09:19.404192    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6bf2e8b_b4f7_412a_a290_228381c5e89f.slice/crio-d6e718605ec1d7dddf7cef72def3cd2a7dadba1c7bed74e6bc73fb5333936bdb WatchSource:0}: Error finding container d6e718605ec1d7dddf7cef72def3cd2a7dadba1c7bed74e6bc73fb5333936bdb: Status 404 returned error can't find the container with id d6e718605ec1d7dddf7cef72def3cd2a7dadba1c7bed74e6bc73fb5333936bdb
Sep 10 15:09:19.417427 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:19.417401    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8" event={"ID":"c6bf2e8b-b4f7-412a-a290-228381c5e89f","Type":"ContainerStarted","Data":"d6e718605ec1d7dddf7cef72def3cd2a7dadba1c7bed74e6bc73fb5333936bdb"}
Sep 10 15:09:20.425550 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:20.425458    2558 generic.go:356] "Generic (PLEG): container finished" podID="c6bf2e8b-b4f7-412a-a290-228381c5e89f" containerID="e2cf711eec120f80aa00f4fa40efb26a38ee255146180ff120ac3da8aa59a2fa" exitCode=0
Sep 10 15:09:20.425955 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:20.425539    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8" event={"ID":"c6bf2e8b-b4f7-412a-a290-228381c5e89f","Type":"ContainerDied","Data":"e2cf711eec120f80aa00f4fa40efb26a38ee255146180ff120ac3da8aa59a2fa"}
Sep 10 15:09:21.564957 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:21.564931    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"
Sep 10 15:09:21.713901 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:21.713869    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf2e8b-b4f7-412a-a290-228381c5e89f-maas-api-service-ca\") pod \"c6bf2e8b-b4f7-412a-a290-228381c5e89f\" (UID: \"c6bf2e8b-b4f7-412a-a290-228381c5e89f\") "
Sep 10 15:09:21.714320 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:21.714292    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6bf2e8b-b4f7-412a-a290-228381c5e89f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6bf2e8b-b4f7-412a-a290-228381c5e89f" (UID: "c6bf2e8b-b4f7-412a-a290-228381c5e89f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:09:21.814967 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:21.814918    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6bf2e8b-b4f7-412a-a290-228381c5e89f-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:09:22.439801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:22.439775    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"
Sep 10 15:09:22.439801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:22.439787    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8" event={"ID":"c6bf2e8b-b4f7-412a-a290-228381c5e89f","Type":"ContainerDied","Data":"d6e718605ec1d7dddf7cef72def3cd2a7dadba1c7bed74e6bc73fb5333936bdb"}
Sep 10 15:09:22.439992 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:22.439839    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6e718605ec1d7dddf7cef72def3cd2a7dadba1c7bed74e6bc73fb5333936bdb"
Sep 10 15:09:25.214164 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:25.214122    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"]
Sep 10 15:09:25.217496 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:25.217471    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-799b5a-tvsp8"]
Sep 10 15:09:26.018326 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:26.018292    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6bf2e8b-b4f7-412a-a290-228381c5e89f" path="/var/lib/kubelet/pods/c6bf2e8b-b4f7-412a-a290-228381c5e89f/volumes"
Sep 10 15:09:29.778844 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:29.778761    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:09:35.604786 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.604747    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"]
Sep 10 15:09:35.605167 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.605157    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6bf2e8b-b4f7-412a-a290-228381c5e89f" containerName="revoke-keys"
Sep 10 15:09:35.605209 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.605170    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6bf2e8b-b4f7-412a-a290-228381c5e89f" containerName="revoke-keys"
Sep 10 15:09:35.605270 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.605260    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6bf2e8b-b4f7-412a-a290-228381c5e89f" containerName="revoke-keys"
Sep 10 15:09:35.608409 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.608392    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"
Sep 10 15:09:35.610956 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.610932    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 15:09:35.611088 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.610935    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:09:35.616591 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.616400    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"]
Sep 10 15:09:35.644597 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.644566    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7acd1df-59f8-44c6-85c3-7a9fd5aafde1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k\" (UID: \"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"
Sep 10 15:09:35.745167 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.745133    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7acd1df-59f8-44c6-85c3-7a9fd5aafde1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k\" (UID: \"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"
Sep 10 15:09:35.745794 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.745771    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7acd1df-59f8-44c6-85c3-7a9fd5aafde1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k\" (UID: \"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"
Sep 10 15:09:35.923023 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.922940    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:09:35.931147 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:35.931127    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"
Sep 10 15:09:36.220273 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:36.220242    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"]
Sep 10 15:09:36.221843 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:09:36.221819    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7acd1df_59f8_44c6_85c3_7a9fd5aafde1.slice/crio-f04f74cb54fe9bece8582bb250fd35b8cb502a01d34525b626d576676e1ab5bf WatchSource:0}: Error finding container f04f74cb54fe9bece8582bb250fd35b8cb502a01d34525b626d576676e1ab5bf: Status 404 returned error can't find the container with id f04f74cb54fe9bece8582bb250fd35b8cb502a01d34525b626d576676e1ab5bf
Sep 10 15:09:36.523128 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:36.523033    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k" event={"ID":"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1","Type":"ContainerStarted","Data":"a4a0c9c8180fad91294934b0c08f53cf206722c776d163b6ba9a1a6a9bb8f3cc"}
Sep 10 15:09:36.523128 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:36.523086    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k" event={"ID":"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1","Type":"ContainerStarted","Data":"f04f74cb54fe9bece8582bb250fd35b8cb502a01d34525b626d576676e1ab5bf"}
Sep 10 15:09:36.539237 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:36.539165    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k" podStartSLOduration=1.539148946 podStartE2EDuration="1.539148946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:09:36.537436499 +0000 UTC m=+1621.024124393" watchObservedRunningTime="2026-09-10 15:09:36.539148946 +0000 UTC m=+1621.025836883"
Sep 10 15:09:37.530558 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:37.530527    2558 generic.go:356] "Generic (PLEG): container finished" podID="b7acd1df-59f8-44c6-85c3-7a9fd5aafde1" containerID="a4a0c9c8180fad91294934b0c08f53cf206722c776d163b6ba9a1a6a9bb8f3cc" exitCode=0
Sep 10 15:09:37.530930 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:37.530607    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k" event={"ID":"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1","Type":"ContainerDied","Data":"a4a0c9c8180fad91294934b0c08f53cf206722c776d163b6ba9a1a6a9bb8f3cc"}
Sep 10 15:09:38.680075 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:38.680049    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"
Sep 10 15:09:38.774608 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:38.774567    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7acd1df-59f8-44c6-85c3-7a9fd5aafde1-maas-api-service-ca\") pod \"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1\" (UID: \"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1\") "
Sep 10 15:09:38.774970 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:38.774945    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7acd1df-59f8-44c6-85c3-7a9fd5aafde1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7acd1df-59f8-44c6-85c3-7a9fd5aafde1" (UID: "b7acd1df-59f8-44c6-85c3-7a9fd5aafde1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:09:38.876038 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:38.875949    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7acd1df-59f8-44c6-85c3-7a9fd5aafde1-maas-api-service-ca\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:09:39.551189 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:39.551157    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"
Sep 10 15:09:39.551362 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:39.551197    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k" event={"ID":"b7acd1df-59f8-44c6-85c3-7a9fd5aafde1","Type":"ContainerDied","Data":"f04f74cb54fe9bece8582bb250fd35b8cb502a01d34525b626d576676e1ab5bf"}
Sep 10 15:09:39.551362 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:39.551267    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f04f74cb54fe9bece8582bb250fd35b8cb502a01d34525b626d576676e1ab5bf"
Sep 10 15:09:48.872473 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:48.872434    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:09:51.499126 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:51.499088    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"]
Sep 10 15:09:51.502737 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:51.502702    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-b022ee-ptb6k"]
Sep 10 15:09:52.019023 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:52.018989    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7acd1df-59f8-44c6-85c3-7a9fd5aafde1" path="/var/lib/kubelet/pods/b7acd1df-59f8-44c6-85c3-7a9fd5aafde1/volumes"
Sep 10 15:09:56.843326 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:09:56.843285    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"]
Sep 10 15:10:02.273892 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:02.273858    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:10:11.863987 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:11.863945    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6" podUID="2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" containerName="main" containerID="cri-o://b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0" gracePeriod=45
Sep 10 15:10:12.111062 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.111034    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:10:12.201694 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.201657    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-home\") pod \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") "
Sep 10 15:10:12.201902 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.201701    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9447\" (UniqueName: \"kubernetes.io/projected/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-kube-api-access-w9447\") pod \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") "
Sep 10 15:10:12.201902 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.201731    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tls-certs\") pod \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") "
Sep 10 15:10:12.201902 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.201839    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tmp-dir\") pod \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") "
Sep 10 15:10:12.201902 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.201902    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-dshm\") pod \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") "
Sep 10 15:10:12.202134 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.201931    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-model-cache\") pod \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\" (UID: \"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3\") "
Sep 10 15:10:12.202134 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.201994    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-home" (OuterVolumeSpecName: "home") pod "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" (UID: "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:12.202134 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.202099    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" (UID: "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:12.202331 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.202250    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-model-cache" (OuterVolumeSpecName: "model-cache") pod "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" (UID: "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:12.202467 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.202398    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-home\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:12.202467 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.202409    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tmp-dir\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:12.202467 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.202419    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-model-cache\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:12.204127 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.204103    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-dshm" (OuterVolumeSpecName: "dshm") pod "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" (UID: "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:10:12.204257 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.204137    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" (UID: "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:10:12.204257 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.204181    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-kube-api-access-w9447" (OuterVolumeSpecName: "kube-api-access-w9447") pod "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" (UID: "2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3"). InnerVolumeSpecName "kube-api-access-w9447". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:10:12.303734 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.303695    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9447\" (UniqueName: \"kubernetes.io/projected/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-kube-api-access-w9447\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:12.303734 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.303727    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-tls-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:12.303734 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.303736    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3-dshm\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:10:12.742956 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.742922    2558 generic.go:356] "Generic (PLEG): container finished" podID="2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" containerID="b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0" exitCode=0
Sep 10 15:10:12.743123 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.742970    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"
Sep 10 15:10:12.743123 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.743004    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6" event={"ID":"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3","Type":"ContainerDied","Data":"b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0"}
Sep 10 15:10:12.743123 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.743054    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6" event={"ID":"2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3","Type":"ContainerDied","Data":"9f34e90f4afe515489faded254e1d6f1a7014d047ab050cc72c3f26861c00c01"}
Sep 10 15:10:12.743123 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.743072    2558 scope.go:117] "RemoveContainer" containerID="b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0"
Sep 10 15:10:12.752890 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.752872    2558 scope.go:117] "RemoveContainer" containerID="b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0"
Sep 10 15:10:12.753164 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:10:12.753149    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0\": container with ID starting with b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0 not found: ID does not exist" containerID="b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0"
Sep 10 15:10:12.753210 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.753173    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0"} err="failed to get container status \"b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0\": rpc error: code = NotFound desc = could not find container \"b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0\": container with ID starting with b451d8e92993b69ae000c3ed35bf74e3225244699fa809da9c194aaa649b93b0 not found: ID does not exist"
Sep 10 15:10:12.764661 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.764634    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"]
Sep 10 15:10:12.768886 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:12.768865    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-b022ee/ipp-route-b022ee-kserve-5b8bd9bcbc-xghb6"]
Sep 10 15:10:14.019297 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:14.019264    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" path="/var/lib/kubelet/pods/2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3/volumes"
Sep 10 15:10:36.348862 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:36.348826    2558 scope.go:117] "RemoveContainer" containerID="010a12e6be097d2366a157a0591147a1d18310513418340d67b9ec62085666ff"
Sep 10 15:10:54.372734 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:54.372684    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:10:58.036047 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:58.035975    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"]
Sep 10 15:10:58.036443 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:10:58.036251    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy" containerID="cri-o://95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212" gracePeriod=30
Sep 10 15:11:03.297296 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.297213    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:11:03.412110 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412076    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-token\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412110 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412122    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-credential-socket\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412148    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-envoy\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412191    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-socket\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412263    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-data\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412286    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcfjr\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-kube-api-access-tcfjr\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412397 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412395    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12fc793b-fb84-4d61-8128-e0d9826d6a14-istiod-ca-cert\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412653 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412435    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-podinfo\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412653 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412459    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-certs\") pod \"12fc793b-fb84-4d61-8128-e0d9826d6a14\" (UID: \"12fc793b-fb84-4d61-8128-e0d9826d6a14\") "
Sep 10 15:11:03.412653 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412484    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:03.412822 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412627    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-data" (OuterVolumeSpecName: "istio-data") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:03.412882 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412816    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-credential-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.412882 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412822    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:03.412882 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412838    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-data\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.412882 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412846    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:03.413043 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.412877    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12fc793b-fb84-4d61-8128-e0d9826d6a14-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:11:03.414678 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.414657    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:03.414678 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.414663    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 15:11:03.415064 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.415045    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-kube-api-access-tcfjr" (OuterVolumeSpecName: "kube-api-access-tcfjr") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "kube-api-access-tcfjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:11:03.415126 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.415050    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-token" (OuterVolumeSpecName: "istio-token") pod "12fc793b-fb84-4d61-8128-e0d9826d6a14" (UID: "12fc793b-fb84-4d61-8128-e0d9826d6a14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:11:03.513797 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.513761    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-envoy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.513797 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.513792    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.513797 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.513801    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcfjr\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-kube-api-access-tcfjr\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.514042 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.513811    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12fc793b-fb84-4d61-8128-e0d9826d6a14-istiod-ca-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.514042 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.513820    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-podinfo\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.514042 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.513827    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12fc793b-fb84-4d61-8128-e0d9826d6a14-workload-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:03.514042 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:03.513835    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12fc793b-fb84-4d61-8128-e0d9826d6a14-istio-token\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:04.054922 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.054887    2558 generic.go:356] "Generic (PLEG): container finished" podID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerID="95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212" exitCode=0
Sep 10 15:11:04.055107 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.054962    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"
Sep 10 15:11:04.055107 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.054964    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" event={"ID":"12fc793b-fb84-4d61-8128-e0d9826d6a14","Type":"ContainerDied","Data":"95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212"}
Sep 10 15:11:04.055107 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.055028    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v" event={"ID":"12fc793b-fb84-4d61-8128-e0d9826d6a14","Type":"ContainerDied","Data":"a3126df29e79db2899442b65cdf6696da79eed230f9271061274738c270d6c8e"}
Sep 10 15:11:04.055107 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.055057    2558 scope.go:117] "RemoveContainer" containerID="95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212"
Sep 10 15:11:04.064933 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.064911    2558 scope.go:117] "RemoveContainer" containerID="e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30"
Sep 10 15:11:04.073851 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.073822    2558 scope.go:117] "RemoveContainer" containerID="95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212"
Sep 10 15:11:04.074362 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:11:04.074329    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212\": container with ID starting with 95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212 not found: ID does not exist" containerID="95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212"
Sep 10 15:11:04.074494 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.074375    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212"} err="failed to get container status \"95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212\": rpc error: code = NotFound desc = could not find container \"95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212\": container with ID starting with 95a31a46f82e456ec149373d1b488461eb8cb5cea4a45dccc0f23eecc8c87212 not found: ID does not exist"
Sep 10 15:11:04.074494 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.074400    2558 scope.go:117] "RemoveContainer" containerID="e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30"
Sep 10 15:11:04.075197 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:11:04.075175    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30\": container with ID starting with e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30 not found: ID does not exist" containerID="e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30"
Sep 10 15:11:04.075391 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.075203    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30"} err="failed to get container status \"e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30\": rpc error: code = NotFound desc = could not find container \"e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30\": container with ID starting with e07c0cf5442aea8f6884a4b22324aa61d0e7b1f3b3f6d1fe4578fd7e6a281f30 not found: ID does not exist"
Sep 10 15:11:04.077320 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.077299    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"]
Sep 10 15:11:04.084068 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:04.084044    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-cb5c8c-openshift-default-75d86674b8-plh9v"]
Sep 10 15:11:06.018807 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:06.018778    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" path="/var/lib/kubelet/pods/12fc793b-fb84-4d61-8128-e0d9826d6a14/volumes"
Sep 10 15:11:11.877380 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:11.877344    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:11:15.907612 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:15.907569    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"]
Sep 10 15:11:15.907991 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:15.907819    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy" containerID="cri-o://e5691eb527d941f8446beb59be84aabfc5831234d2cf28553e752fb98711cfe4" gracePeriod=30
Sep 10 15:11:21.153694 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.153661    2558 generic.go:356] "Generic (PLEG): container finished" podID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerID="e5691eb527d941f8446beb59be84aabfc5831234d2cf28553e752fb98711cfe4" exitCode=0
Sep 10 15:11:21.154046 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.153732    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" event={"ID":"33bd8417-22b4-4689-969a-23fb724f9ee3","Type":"ContainerDied","Data":"e5691eb527d941f8446beb59be84aabfc5831234d2cf28553e752fb98711cfe4"}
Sep 10 15:11:21.154046 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.153793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9" event={"ID":"33bd8417-22b4-4689-969a-23fb724f9ee3","Type":"ContainerDied","Data":"04559d51734a2be1408da620d16c2d43faef2b73f608d8e615295ffd4ccc0007"}
Sep 10 15:11:21.154046 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.153810    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04559d51734a2be1408da620d16c2d43faef2b73f608d8e615295ffd4ccc0007"
Sep 10 15:11:21.154046 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.153823    2558 scope.go:117] "RemoveContainer" containerID="5412e678a174ae076f200d7b477ddc5d5283368f21432ab48f7d3446a6ac11da"
Sep 10 15:11:21.162833 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.162811    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 15:11:21.182446 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182384    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd8417-22b4-4689-969a-23fb724f9ee3-istiod-ca-cert\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182446 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182417    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-socket\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182454    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-token\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182476    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-podinfo\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182498    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-data\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182526    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-credential-socket\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182570    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7fl8\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-kube-api-access-m7fl8\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182644 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182627    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-certs\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182951 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182675    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-envoy\") pod \"33bd8417-22b4-4689-969a-23fb724f9ee3\" (UID: \"33bd8417-22b4-4689-969a-23fb724f9ee3\") "
Sep 10 15:11:21.182951 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182816    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33bd8417-22b4-4689-969a-23fb724f9ee3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:11:21.183063 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.182986    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:21.183118 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.183060    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-data" (OuterVolumeSpecName: "istio-data") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:21.183118 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.183082    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:21.183244 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.183156    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-data\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.183244 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.183174    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd8417-22b4-4689-969a-23fb724f9ee3-istiod-ca-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.183244 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.183189    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.183558 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.183342    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:21.185924 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.185896    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 15:11:21.186236 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.186192    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:21.187025 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.186989    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-kube-api-access-m7fl8" (OuterVolumeSpecName: "kube-api-access-m7fl8") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "kube-api-access-m7fl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:11:21.187135 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.187074    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-token" (OuterVolumeSpecName: "istio-token") pod "33bd8417-22b4-4689-969a-23fb724f9ee3" (UID: "33bd8417-22b4-4689-969a-23fb724f9ee3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:11:21.283801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.283766    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-podinfo\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.283801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.283797    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-credential-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.283801 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.283807    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7fl8\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-kube-api-access-m7fl8\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.284041 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.283816    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-workload-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.284041 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.283824    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-envoy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:21.284041 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:21.283832    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd8417-22b4-4689-969a-23fb724f9ee3-istio-token\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:22.160889 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:22.160858    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"
Sep 10 15:11:22.208579 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:22.208537    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"]
Sep 10 15:11:22.212621 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:22.212594    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-16ad45-openshift-default-7595944c94-vgmp9"]
Sep 10 15:11:24.018646 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:24.018609    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" path="/var/lib/kubelet/pods/33bd8417-22b4-4689-969a-23fb724f9ee3/volumes"
Sep 10 15:11:29.373229 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:29.373175    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:11:34.329398 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:34.329366    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"]
Sep 10 15:11:34.329794 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:34.329619    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy" containerID="cri-o://55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5" gracePeriod=30
Sep 10 15:11:36.376706 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:36.376673    2558 scope.go:117] "RemoveContainer" containerID="585f9521bd15642d3b45f7f69943d46cfaf67ba3c4b20027bf4e3ac2515bee7f"
Sep 10 15:11:36.385555 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:36.385535    2558 scope.go:117] "RemoveContainer" containerID="e5691eb527d941f8446beb59be84aabfc5831234d2cf28553e752fb98711cfe4"
Sep 10 15:11:38.573408 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:38.573354    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:11:39.587654 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.587625    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:11:39.663250 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663138    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-credential-socket\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663250 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663197    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-socket\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663250 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663230    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-envoy\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663250 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663247    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrqzg\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-kube-api-access-hrqzg\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663293    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65ad5b87-68ce-4bae-8702-05416361c76f-istiod-ca-cert\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663318    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-data\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663356    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-certs\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663394    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65ad5b87-68ce-4bae-8702-05416361c76f-istio-podinfo\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663415    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-istio-token\") pod \"65ad5b87-68ce-4bae-8702-05416361c76f\" (UID: \"65ad5b87-68ce-4bae-8702-05416361c76f\") "
Sep 10 15:11:39.663587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663493    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:39.663587 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663538    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:39.663938 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663646    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:39.663938 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663717    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65ad5b87-68ce-4bae-8702-05416361c76f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:11:39.663938 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663743    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.663938 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663762    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-workload-certs\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.663938 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663777    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-credential-socket\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.663938 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.663835    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-data" (OuterVolumeSpecName: "istio-data") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:39.665702 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.665674    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-istio-token" (OuterVolumeSpecName: "istio-token") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:11:39.665848 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.665746    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-kube-api-access-hrqzg" (OuterVolumeSpecName: "kube-api-access-hrqzg") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "kube-api-access-hrqzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:11:39.666103 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.666083    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65ad5b87-68ce-4bae-8702-05416361c76f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 15:11:39.666162 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.666087    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65ad5b87-68ce-4bae-8702-05416361c76f" (UID: "65ad5b87-68ce-4bae-8702-05416361c76f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 15:11:39.764711 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.764674    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-envoy\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.764711 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.764704    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrqzg\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-kube-api-access-hrqzg\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.764910 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.764718    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65ad5b87-68ce-4bae-8702-05416361c76f-istiod-ca-cert\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.764910 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.764733    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65ad5b87-68ce-4bae-8702-05416361c76f-istio-data\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.764910 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.764741    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65ad5b87-68ce-4bae-8702-05416361c76f-istio-podinfo\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:39.764910 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:39.764748    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65ad5b87-68ce-4bae-8702-05416361c76f-istio-token\") on node \"ip-10-0-142-253.ec2.internal\" DevicePath \"\""
Sep 10 15:11:40.262532 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.262494    2558 generic.go:356] "Generic (PLEG): container finished" podID="65ad5b87-68ce-4bae-8702-05416361c76f" containerID="55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5" exitCode=0
Sep 10 15:11:40.262678 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.262540    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"
Sep 10 15:11:40.262678 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.262583    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" event={"ID":"65ad5b87-68ce-4bae-8702-05416361c76f","Type":"ContainerDied","Data":"55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5"}
Sep 10 15:11:40.262678 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.262644    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g" event={"ID":"65ad5b87-68ce-4bae-8702-05416361c76f","Type":"ContainerDied","Data":"72cd867820d4611ed1b353903467ec24cafc2adf668734196338e9fec7abebc0"}
Sep 10 15:11:40.262678 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.262669    2558 scope.go:117] "RemoveContainer" containerID="55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5"
Sep 10 15:11:40.272660 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.272644    2558 scope.go:117] "RemoveContainer" containerID="6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3"
Sep 10 15:11:40.281174 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.281156    2558 scope.go:117] "RemoveContainer" containerID="55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5"
Sep 10 15:11:40.281494 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:11:40.281470    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5\": container with ID starting with 55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5 not found: ID does not exist" containerID="55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5"
Sep 10 15:11:40.281570 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.281506    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5"} err="failed to get container status \"55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5\": rpc error: code = NotFound desc = could not find container \"55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5\": container with ID starting with 55710643eba791836853d7f5332139b8ebc8003bb7cd2f96cda7caa09835c2e5 not found: ID does not exist"
Sep 10 15:11:40.281570 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.281530    2558 scope.go:117] "RemoveContainer" containerID="6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3"
Sep 10 15:11:40.281832 ip-10-0-142-253 kubenswrapper[2558]: E0910 15:11:40.281815    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3\": container with ID starting with 6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3 not found: ID does not exist" containerID="6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3"
Sep 10 15:11:40.281897 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.281837    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3"} err="failed to get container status \"6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3\": rpc error: code = NotFound desc = could not find container \"6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3\": container with ID starting with 6d2fbce2a5d5421e14f551a99636c2c7a747c448ef862c5d94d4ac69e2d713d3 not found: ID does not exist"
Sep 10 15:11:40.283111 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.283094    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"]
Sep 10 15:11:40.291201 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:40.291168    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-3f81b7-openshift-default-6ddc4f48b8-fh26g"]
Sep 10 15:11:42.019055 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:11:42.019019    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" path="/var/lib/kubelet/pods/65ad5b87-68ce-4bae-8702-05416361c76f/volumes"
Sep 10 15:13:01.475066 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:13:01.475030    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:13:28.373733 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:13:28.373699    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:13:36.074179 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:13:36.074145    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:13:44.464902 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:13:44.464868    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:14:02.181191 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:14:02.181113    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:14:08.174508 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:14:08.174472    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:15:00.138090 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138049    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8"]
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138652    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138674    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138694    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138703    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138714    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7acd1df-59f8-44c6-85c3-7a9fd5aafde1" containerName="revoke-keys"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138722    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7acd1df-59f8-44c6-85c3-7a9fd5aafde1" containerName="revoke-keys"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138734    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138743    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138751    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138759    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy"
Sep 10 15:15:00.138762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138770    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138777    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138792    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138801    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138819    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" containerName="main"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138828    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" containerName="main"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138928    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138940    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7acd1df-59f8-44c6-85c3-7a9fd5aafde1" containerName="revoke-keys"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138953    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138962    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bd8417-22b4-4689-969a-23fb724f9ee3" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138970    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fe53ddb-8fdf-4d8e-a553-9a614adc9ce3" containerName="main"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138978    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy"
Sep 10 15:15:00.139346 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.138986    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="12fc793b-fb84-4d61-8128-e0d9826d6a14" containerName="istio-proxy"
Sep 10 15:15:00.142422 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.142401    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8"
Sep 10 15:15:00.145569 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.145531    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\""
Sep 10 15:15:00.147308 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.147283    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8"]
Sep 10 15:15:00.154017 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.153993    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8"
Sep 10 15:15:00.426599 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:00.426572    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8"]
Sep 10 15:15:00.428177 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:15:00.428154    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod383660e6_ebad_432d_bafe_2e054be02d77.slice/crio-06aa3b621830d0e19cb73f7fb6b18da677ad76462806a50b896ab306cf7350b1 WatchSource:0}: Error finding container 06aa3b621830d0e19cb73f7fb6b18da677ad76462806a50b896ab306cf7350b1: Status 404 returned error can't find the container with id 06aa3b621830d0e19cb73f7fb6b18da677ad76462806a50b896ab306cf7350b1
Sep 10 15:15:01.388326 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:01.388293    2558 generic.go:356] "Generic (PLEG): container finished" podID="383660e6-ebad-432d-bafe-2e054be02d77" containerID="6bb7155eab6c0fa049532e2258431a09d88cc02fcbf2c2bb601dc85689f9bfd7" exitCode=0
Sep 10 15:15:01.388671 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:01.388346    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8" event={"ID":"383660e6-ebad-432d-bafe-2e054be02d77","Type":"ContainerDied","Data":"6bb7155eab6c0fa049532e2258431a09d88cc02fcbf2c2bb601dc85689f9bfd7"}
Sep 10 15:15:01.388671 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:01.388396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8" event={"ID":"383660e6-ebad-432d-bafe-2e054be02d77","Type":"ContainerStarted","Data":"06aa3b621830d0e19cb73f7fb6b18da677ad76462806a50b896ab306cf7350b1"}
Sep 10 15:15:02.526758 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:02.526727    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8"
Sep 10 15:15:03.400271 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:03.400206    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8" event={"ID":"383660e6-ebad-432d-bafe-2e054be02d77","Type":"ContainerDied","Data":"06aa3b621830d0e19cb73f7fb6b18da677ad76462806a50b896ab306cf7350b1"}
Sep 10 15:15:03.400478 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:03.400292    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06aa3b621830d0e19cb73f7fb6b18da677ad76462806a50b896ab306cf7350b1"
Sep 10 15:15:03.400478 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:03.400246    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817555-nhxd8"
Sep 10 15:15:33.381533 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:33.381312    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:15:36.444839 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:36.444806    2558 scope.go:117] "RemoveContainer" containerID="e2cf711eec120f80aa00f4fa40efb26a38ee255146180ff120ac3da8aa59a2fa"
Sep 10 15:15:36.453568 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:36.453548    2558 scope.go:117] "RemoveContainer" containerID="a4a0c9c8180fad91294934b0c08f53cf206722c776d163b6ba9a1a6a9bb8f3cc"
Sep 10 15:15:39.071912 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:39.071876    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:15:47.377542 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:15:47.377509    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:16:07.773213 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:16:07.773177    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:16:23.268884 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:16:23.268849    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:16:36.073181 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:16:36.073146    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:16:43.573004 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:16:43.572964    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:16:50.972133 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:16:50.972101    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:16:58.796579 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:16:58.796502    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:17:05.472545 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:05.472509    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:17:11.374212 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:11.374156    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:17:18.474008 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:18.473954    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-5smxx"]
Sep 10 15:17:21.174929 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:21.174898    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-795b565bc7-s2f8c_0bbdb5d7-2872-4e0f-89d0-22589b15621f/authorino/0.log"
Sep 10 15:17:27.239854 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:27.239824    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-lrlkh_1a77a12a-da84-4e91-a891-a7b9805dd875/copy-manifests/0.log"
Sep 10 15:17:27.245727 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:27.245703    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-lrlkh_1a77a12a-da84-4e91-a891-a7b9805dd875/manager/0.log"
Sep 10 15:17:28.041711 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:28.041674    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817555-nhxd8_383660e6-ebad-432d-bafe-2e054be02d77/cleanup/0.log"
Sep 10 15:17:28.916657 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:28.916621    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-795b565bc7-s2f8c_0bbdb5d7-2872-4e0f-89d0-22589b15621f/authorino/0.log"
Sep 10 15:17:29.256732 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:29.256706    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-bfzt6_59004aaa-96d5-4b27-a54f-003b649dfb36/kuadrant-console-plugin/0.log"
Sep 10 15:17:29.504396 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:29.504362    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-5smxx_680ffc7c-90dd-4fef-b8d4-1b978a41a79b/limitador/0.log"
Sep 10 15:17:30.331344 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:30.331315    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b7fd47cd-cgvr7_d187a2fb-7de0-4471-b2b1-77e8e936ba89/kube-auth-proxy/0.log"
Sep 10 15:17:31.124550 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:31.124512    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jbcwc_7e8a3ce1-9ec9-435b-83e7-442aa0ce8f64/main/0.log"
Sep 10 15:17:31.344431 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:31.344386    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-8sn2c_4117a54b-cb10-49e9-9ad2-9056d26f93d0/main/0.log"
Sep 10 15:17:31.795781 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:31.795752    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-wd8dv_f5531299-f326-4594-a2ac-be4d52a0c789/main/0.log"
Sep 10 15:17:37.136911 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.136873    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prcmw/must-gather-kcm7x"]
Sep 10 15:17:37.139003 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.137285    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="383660e6-ebad-432d-bafe-2e054be02d77" containerName="cleanup"
Sep 10 15:17:37.139003 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.137299    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="383660e6-ebad-432d-bafe-2e054be02d77" containerName="cleanup"
Sep 10 15:17:37.139003 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.137370    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="65ad5b87-68ce-4bae-8702-05416361c76f" containerName="istio-proxy"
Sep 10 15:17:37.139003 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.137378    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="383660e6-ebad-432d-bafe-2e054be02d77" containerName="cleanup"
Sep 10 15:17:37.140333 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.140308    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.143477 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.143455    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prcmw\"/\"kube-root-ca.crt\""
Sep 10 15:17:37.143591 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.143455    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prcmw\"/\"openshift-service-ca.crt\""
Sep 10 15:17:37.143591 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.143535    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-prcmw\"/\"default-dockercfg-9v44h\""
Sep 10 15:17:37.151165 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.151141    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prcmw/must-gather-kcm7x"]
Sep 10 15:17:37.266894 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.266857    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8b3d844-3cf0-4bb8-9d2c-84ee922ef434-must-gather-output\") pod \"must-gather-kcm7x\" (UID: \"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434\") " pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.267076 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.267014    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzjpm\" (UniqueName: \"kubernetes.io/projected/c8b3d844-3cf0-4bb8-9d2c-84ee922ef434-kube-api-access-nzjpm\") pod \"must-gather-kcm7x\" (UID: \"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434\") " pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.368340 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.368303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzjpm\" (UniqueName: \"kubernetes.io/projected/c8b3d844-3cf0-4bb8-9d2c-84ee922ef434-kube-api-access-nzjpm\") pod \"must-gather-kcm7x\" (UID: \"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434\") " pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.368513 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.368349    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8b3d844-3cf0-4bb8-9d2c-84ee922ef434-must-gather-output\") pod \"must-gather-kcm7x\" (UID: \"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434\") " pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.368720 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.368699    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c8b3d844-3cf0-4bb8-9d2c-84ee922ef434-must-gather-output\") pod \"must-gather-kcm7x\" (UID: \"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434\") " pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.377017 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.376989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzjpm\" (UniqueName: \"kubernetes.io/projected/c8b3d844-3cf0-4bb8-9d2c-84ee922ef434-kube-api-access-nzjpm\") pod \"must-gather-kcm7x\" (UID: \"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434\") " pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.450554 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.450518    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prcmw/must-gather-kcm7x"
Sep 10 15:17:37.725385 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.725360    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prcmw/must-gather-kcm7x"]
Sep 10 15:17:37.727466 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:17:37.727436    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8b3d844_3cf0_4bb8_9d2c_84ee922ef434.slice/crio-7b3a48e1224f0bb987adc3ed5b737d8fa2207a46f14c8ee8c5366fd6bc737b72 WatchSource:0}: Error finding container 7b3a48e1224f0bb987adc3ed5b737d8fa2207a46f14c8ee8c5366fd6bc737b72: Status 404 returned error can't find the container with id 7b3a48e1224f0bb987adc3ed5b737d8fa2207a46f14c8ee8c5366fd6bc737b72
Sep 10 15:17:37.729420 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:37.729403    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 15:17:38.248525 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:38.248485    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prcmw/must-gather-kcm7x" event={"ID":"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434","Type":"ContainerStarted","Data":"7b3a48e1224f0bb987adc3ed5b737d8fa2207a46f14c8ee8c5366fd6bc737b72"}
Sep 10 15:17:39.261877 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:39.261822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prcmw/must-gather-kcm7x" event={"ID":"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434","Type":"ContainerStarted","Data":"f53aba99dff3f334cb6a9d9e1774374927f8eb4e6ce58650fbead5cb55afd037"}
Sep 10 15:17:39.262372 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:39.261890    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prcmw/must-gather-kcm7x" event={"ID":"c8b3d844-3cf0-4bb8-9d2c-84ee922ef434","Type":"ContainerStarted","Data":"f9762c7adb3389a6857b616933d24f4c794706f8a5a2b3f60627121c83c63380"}
Sep 10 15:17:39.280266 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:39.280182    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prcmw/must-gather-kcm7x" podStartSLOduration=1.535669397 podStartE2EDuration="2.280163501s" totalImagesPullingTime="744.494104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:39.279055258 +0000 UTC m=+2103.765743151" watchObservedRunningTime="2026-09-10 15:17:39.280163501 +0000 UTC m=+2103.766851396"
Sep 10 15:17:40.070631 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:40.070600    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g7dg8_c1938126-5194-491b-b8e3-a602e268c691/global-pull-secret-syncer/0.log"
Sep 10 15:17:40.269852 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:40.269805    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vp8pc_de043964-4505-4d6c-9af0-62ba8b338a9d/konnectivity-agent/0.log"
Sep 10 15:17:40.346888 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:40.346811    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-253.ec2.internal_6ff70c42461e7f0c1049ab3f77cb139d/haproxy/0.log"
Sep 10 15:17:45.062716 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:45.062468    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-795b565bc7-s2f8c_0bbdb5d7-2872-4e0f-89d0-22589b15621f/authorino/0.log"
Sep 10 15:17:45.158319 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:45.158287    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-bfzt6_59004aaa-96d5-4b27-a54f-003b649dfb36/kuadrant-console-plugin/0.log"
Sep 10 15:17:45.240460 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:45.240424    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-5smxx_680ffc7c-90dd-4fef-b8d4-1b978a41a79b/limitador/0.log"
Sep 10 15:17:46.721891 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.721864    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363/alertmanager/0.log"
Sep 10 15:17:46.762940 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.762910    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363/config-reloader/0.log"
Sep 10 15:17:46.785085 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.785056    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363/kube-rbac-proxy-web/0.log"
Sep 10 15:17:46.808907 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.808881    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363/kube-rbac-proxy/0.log"
Sep 10 15:17:46.832405 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.832369    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363/kube-rbac-proxy-metric/0.log"
Sep 10 15:17:46.856770 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.856743    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363/prom-label-proxy/0.log"
Sep 10 15:17:46.888598 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.888558    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fb4a63ef-f88e-4d50-b8e3-5d3c4c85f363/init-config-reloader/0.log"
Sep 10 15:17:46.964248 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.964201    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-k4nzs_34467f13-07d0-4d86-bdec-caa7c2098e84/kube-state-metrics/0.log"
Sep 10 15:17:46.985616 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:46.985531    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-k4nzs_34467f13-07d0-4d86-bdec-caa7c2098e84/kube-rbac-proxy-main/0.log"
Sep 10 15:17:47.020928 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.020873    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-k4nzs_34467f13-07d0-4d86-bdec-caa7c2098e84/kube-rbac-proxy-self/0.log"
Sep 10 15:17:47.091381 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.091353    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7f5544f95d-699pr_9e23cf93-6e32-4968-8886-3d7e58600b4a/metrics-server/0.log"
Sep 10 15:17:47.176959 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.176930    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-h4ps8_c2dfa7b0-b5cc-4a78-a442-b157cefdb26a/monitoring-plugin/0.log"
Sep 10 15:17:47.236752 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.236680    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89hhw_ca37333a-5822-44cd-8482-e673c1cb7d18/node-exporter/0.log"
Sep 10 15:17:47.288018 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.287991    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89hhw_ca37333a-5822-44cd-8482-e673c1cb7d18/kube-rbac-proxy/0.log"
Sep 10 15:17:47.356679 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.356638    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-89hhw_ca37333a-5822-44cd-8482-e673c1cb7d18/init-textfile/0.log"
Sep 10 15:17:47.609254 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.609150    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-vwnfk_35527307-227f-4f31-a7e0-fa2c7fa7e13d/kube-rbac-proxy-main/0.log"
Sep 10 15:17:47.631929 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.631893    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-vwnfk_35527307-227f-4f31-a7e0-fa2c7fa7e13d/kube-rbac-proxy-self/0.log"
Sep 10 15:17:47.653890 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.653856    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-vwnfk_35527307-227f-4f31-a7e0-fa2c7fa7e13d/openshift-state-metrics/0.log"
Sep 10 15:17:47.862538 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.862449    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-zmfb2_173b337b-ac39-405d-aab9-06d3b35fb3a7/prometheus-operator/0.log"
Sep 10 15:17:47.895724 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.895663    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-zmfb2_173b337b-ac39-405d-aab9-06d3b35fb3a7/kube-rbac-proxy/0.log"
Sep 10 15:17:47.957102 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:47.957067    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5855b59cc9-g2fvt_4f9bcdb1-9d2b-41d2-8816-62117086a9c6/telemeter-client/0.log"
Sep 10 15:17:48.004591 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.004561    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5855b59cc9-g2fvt_4f9bcdb1-9d2b-41d2-8816-62117086a9c6/reload/0.log"
Sep 10 15:17:48.052492 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.052460    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5855b59cc9-g2fvt_4f9bcdb1-9d2b-41d2-8816-62117086a9c6/kube-rbac-proxy/0.log"
Sep 10 15:17:48.118425 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.118355    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f98cb55f6-gkjjq_85c7d4f8-52fc-498f-91e5-bf1dab056856/thanos-query/0.log"
Sep 10 15:17:48.157020 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.156991    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f98cb55f6-gkjjq_85c7d4f8-52fc-498f-91e5-bf1dab056856/kube-rbac-proxy-web/0.log"
Sep 10 15:17:48.178880 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.178851    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f98cb55f6-gkjjq_85c7d4f8-52fc-498f-91e5-bf1dab056856/kube-rbac-proxy/0.log"
Sep 10 15:17:48.205460 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.205391    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f98cb55f6-gkjjq_85c7d4f8-52fc-498f-91e5-bf1dab056856/prom-label-proxy/0.log"
Sep 10 15:17:48.228723 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.228694    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f98cb55f6-gkjjq_85c7d4f8-52fc-498f-91e5-bf1dab056856/kube-rbac-proxy-rules/0.log"
Sep 10 15:17:48.250739 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:48.250709    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f98cb55f6-gkjjq_85c7d4f8-52fc-498f-91e5-bf1dab056856/kube-rbac-proxy-metrics/0.log"
Sep 10 15:17:50.325873 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.325829    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"]
Sep 10 15:17:50.334198 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.334167    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.342097 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.342070    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"]
Sep 10 15:17:50.414632 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.414578    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-proc\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.414632 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.414633    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-sys\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.414939 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.414736    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6dvm\" (UniqueName: \"kubernetes.io/projected/1529e25a-84b3-42a3-90c7-d51d8412806b-kube-api-access-f6dvm\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.414939 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.414775    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-podres\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.414939 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.414800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-lib-modules\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.516468 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.516419    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6dvm\" (UniqueName: \"kubernetes.io/projected/1529e25a-84b3-42a3-90c7-d51d8412806b-kube-api-access-f6dvm\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.516762 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.516730    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-podres\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.517392 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.517368    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-lib-modules\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.517637 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.517623    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-proc\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.517780 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.517767    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-sys\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.518034 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.518006    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-sys\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.518152 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.516991    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-podres\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.518415 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.518400    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-lib-modules\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.518603 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.518574    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1529e25a-84b3-42a3-90c7-d51d8412806b-proc\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.526868 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.526830    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6dvm\" (UniqueName: \"kubernetes.io/projected/1529e25a-84b3-42a3-90c7-d51d8412806b-kube-api-access-f6dvm\") pod \"perf-node-gather-daemonset-9h2wt\" (UID: \"1529e25a-84b3-42a3-90c7-d51d8412806b\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.650139 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.650057    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:50.705819 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.705777    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-556854d599-9hkpg_c2f5c934-5c72-4f90-a52a-ac0fae6c208d/console/0.log"
Sep 10 15:17:50.955646 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:50.955611    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"]
Sep 10 15:17:50.958839 ip-10-0-142-253 kubenswrapper[2558]: W0910 15:17:50.958804    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1529e25a_84b3_42a3_90c7_d51d8412806b.slice/crio-db1838d3fd71038f076186cb78ee76ede336efab40bc5b98f91b2db1307cdd9f WatchSource:0}: Error finding container db1838d3fd71038f076186cb78ee76ede336efab40bc5b98f91b2db1307cdd9f: Status 404 returned error can't find the container with id db1838d3fd71038f076186cb78ee76ede336efab40bc5b98f91b2db1307cdd9f
Sep 10 15:17:51.373296 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:51.373184    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt" event={"ID":"1529e25a-84b3-42a3-90c7-d51d8412806b","Type":"ContainerStarted","Data":"b264edf8b9cbe23b000ecd42c11ca8399ccced470e218fefdedf434d18f0ebef"}
Sep 10 15:17:51.373296 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:51.373280    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt" event={"ID":"1529e25a-84b3-42a3-90c7-d51d8412806b","Type":"ContainerStarted","Data":"db1838d3fd71038f076186cb78ee76ede336efab40bc5b98f91b2db1307cdd9f"}
Sep 10 15:17:51.374129 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:51.374103    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:52.563943 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:52.563902    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bx5ht_80829f4e-d9fa-4a86-9d34-371483048e09/dns/0.log"
Sep 10 15:17:52.586197 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:52.586169    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bx5ht_80829f4e-d9fa-4a86-9d34-371483048e09/kube-rbac-proxy/0.log"
Sep 10 15:17:52.753335 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:52.753299    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qglhm_523956a8-c95c-44d7-8a58-48997ea2f395/dns-node-resolver/0.log"
Sep 10 15:17:53.377673 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:53.377644    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nmjz6_11d4c95f-8587-43ee-addb-325df5853d41/node-ca/0.log"
Sep 10 15:17:54.620911 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:54.620878    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b7fd47cd-cgvr7_d187a2fb-7de0-4471-b2b1-77e8e936ba89/kube-auth-proxy/0.log"
Sep 10 15:17:55.651328 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:55.651300    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j82z2_43d5b4a5-d9d8-4a8b-bda9-0cb39deaab6d/serve-healthcheck-canary/0.log"
Sep 10 15:17:56.290721 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:56.290695    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wh2cr_eb1b0606-9efe-4e97-8a0e-b06d4eecf18b/kube-rbac-proxy/0.log"
Sep 10 15:17:56.315844 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:56.315779    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wh2cr_eb1b0606-9efe-4e97-8a0e-b06d4eecf18b/exporter/0.log"
Sep 10 15:17:56.340408 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:56.340383    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wh2cr_eb1b0606-9efe-4e97-8a0e-b06d4eecf18b/extractor/0.log"
Sep 10 15:17:58.398109 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:58.398080    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt"
Sep 10 15:17:58.415426 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:58.415359    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-9h2wt" podStartSLOduration=8.415338519 podStartE2EDuration="8.415338519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:51.394412123 +0000 UTC m=+2115.881100018" watchObservedRunningTime="2026-09-10 15:17:58.415338519 +0000 UTC m=+2122.902026413"
Sep 10 15:17:58.948146 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:58.948118    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-lrlkh_1a77a12a-da84-4e91-a891-a7b9805dd875/manager/0.log"
Sep 10 15:17:58.974460 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:17:58.974420    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-lrlkh_1a77a12a-da84-4e91-a891-a7b9805dd875/copy-manifests/0.log"
Sep 10 15:18:00.576678 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:00.576639    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bc768b6-qm2tn_83dd1085-7481-4cd4-9762-09e3758faa04/manager/0.log"
Sep 10 15:18:01.169944 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:01.169914    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b7fd47cd-cgvr7_d187a2fb-7de0-4471-b2b1-77e8e936ba89/kube-auth-proxy/0.log"
Sep 10 15:18:02.957163 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:02.957094    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79b7fd47cd-cgvr7_d187a2fb-7de0-4471-b2b1-77e8e936ba89/kube-auth-proxy/0.log"
Sep 10 15:18:07.189333 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.189307    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8nc57_0a44e73c-8b44-4d88-97b0-ce0446954353/kube-multus-additional-cni-plugins/0.log"
Sep 10 15:18:07.213592 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.213566    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8nc57_0a44e73c-8b44-4d88-97b0-ce0446954353/egress-router-binary-copy/0.log"
Sep 10 15:18:07.237751 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.237723    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8nc57_0a44e73c-8b44-4d88-97b0-ce0446954353/cni-plugins/0.log"
Sep 10 15:18:07.260426 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.260390    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8nc57_0a44e73c-8b44-4d88-97b0-ce0446954353/bond-cni-plugin/0.log"
Sep 10 15:18:07.283129 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.283102    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8nc57_0a44e73c-8b44-4d88-97b0-ce0446954353/routeoverride-cni/0.log"
Sep 10 15:18:07.305479 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.305453    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8nc57_0a44e73c-8b44-4d88-97b0-ce0446954353/whereabouts-cni-bincopy/0.log"
Sep 10 15:18:07.329124 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.329093    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8nc57_0a44e73c-8b44-4d88-97b0-ce0446954353/whereabouts-cni/0.log"
Sep 10 15:18:07.833962 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.833930    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zfx7h_d36866a7-ce48-4c5f-b14a-b3cad558f061/kube-multus/0.log"
Sep 10 15:18:07.856950 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.856917    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hbxnv_0fb75b7e-b7c0-4bfa-8c51-9b96e4449391/network-metrics-daemon/0.log"
Sep 10 15:18:07.880373 ip-10-0-142-253 kubenswrapper[2558]: I0910 15:18:07.880341    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hbxnv_0fb75b7e-b7c0-4bfa-8c51-9b96e4449391/kube-rbac-proxy/0.log"