Sep 08 15:44:48.916605 ip-10-0-136-224 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 15:44:49.276461 ip-10-0-136-224 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:44:49.276461 ip-10-0-136-224 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 15:44:49.276461 ip-10-0-136-224 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:44:49.276461 ip-10-0-136-224 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 15:44:49.276461 ip-10-0-136-224 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:44:49.277458 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.277368    2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 15:44:49.281483 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281468    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:44:49.281483 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281483    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281487    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281492    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281496    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281499    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281503    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281505    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281508    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281511    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281514    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281517    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281520    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281523    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281526    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281529    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281532    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281534    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281537    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281539    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281542    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:44:49.281545 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281545    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281547    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281550    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281553    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281556    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281559    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281562    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281564    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281567    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281569    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281572    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281575    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281577    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281580    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281582    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281585    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281589    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281593    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281596    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:44:49.282051 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281599    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281602    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281604    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281607    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281610    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281614    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281618    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281621    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281624    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281626    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281629    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281632    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281635    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281637    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281640    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281643    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281645    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281648    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281652    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:44:49.282543 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281655    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281658    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281661    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281663    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281666    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281669    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281672    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281674    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281677    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281680    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281682    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281685    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281689    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281693    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281696    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281699    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281702    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281705    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281708    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:44:49.283013 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281711    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281715    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281718    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281720    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281723    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281726    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281728    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281731    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281733    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281736    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281739    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281741    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281744    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281747    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281749    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281752    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281755    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281757    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281760    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281763    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:44:49.283488 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.281765    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282469    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282476    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282479    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282482    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282485    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282488    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282491    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282494    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282497    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282499    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282502    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282505    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282508    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282510    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282513    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282516    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282518    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282521    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282523    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:44:49.283968 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282526    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282528    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282531    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282533    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282536    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282538    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282541    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282544    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282546    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282549    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282552    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282555    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282557    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282560    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282570    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282572    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282577    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282581    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282584    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282588    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:44:49.284522 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282591    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282594    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282596    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282601    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282604    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282606    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282609    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282612    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282615    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282617    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282620    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282622    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282625    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282627    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282630    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282632    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282635    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282637    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282640    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282643    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:44:49.285011 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282645    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282648    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282651    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282653    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282656    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282659    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282661    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282665    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282668    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282671    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282675    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282678    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282682    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282685    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282687    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282691    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282694    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282696    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282699    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:44:49.285516 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282703    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282707    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282710    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282713    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282716    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282720    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282722    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282725    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282727    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282730    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282732    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282735    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282737    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282740    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282742    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282745    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282748    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282750    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282753    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282755    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:44:49.285971 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.282758    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283122    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283127    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283130    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283133    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283136    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283139    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283142    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283146    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283150    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283152    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283155    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283158    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283161    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283163    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283166    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283169    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283171    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283174    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283176    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:44:49.286475 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283180    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283182    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283185    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283188    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283190    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283193    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283195    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283198    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283201    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283204    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283206    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283209    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283212    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283214    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283217    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283219    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283222    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283225    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283228    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283231    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:44:49.287047 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283234    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283236    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283239    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283242    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283246    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283249    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283252    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283255    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283258    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283261    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283263    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283267    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283270    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283273    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283276    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283279    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283282    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283285    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:44:49.287537 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283287    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283290    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283293    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283296    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283299    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283303    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283306    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283309    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283325    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283328    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283330    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283333    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283335    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283339    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283341    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283345    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283347    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283350    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283352    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283355    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:44:49.287976 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283358    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283360    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283364    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283366    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283369    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283372    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283374    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283376    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283379    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283382    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283384    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283387    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283389    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283392    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283394    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283397    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283399    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283403    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283405    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283408    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:44:49.288501 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283411    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.283413    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283499    2552 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283550    2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283558    2552 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283563    2552 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283568    2552 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283572    2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283576    2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283581    2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283584    2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283587    2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283590    2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283593    2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283596    2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283599    2552 flags.go:64] FLAG: --cgroup-root=""
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283602    2552 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283605    2552 flags.go:64] FLAG: --client-ca-file=""
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283608    2552 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283612    2552 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283616    2552 flags.go:64] FLAG: --cluster-domain=""
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283619    2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283622    2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283625    2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 15:44:49.288969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283628    2552 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283632    2552 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283635    2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283638    2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283643    2552 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283649    2552 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283651    2552 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283654    2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283657    2552 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283660    2552 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283665    2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283668    2552 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283671    2552 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283673    2552 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283677    2552 flags.go:64] FLAG: --enable-server="true"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283680    2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283684    2552 flags.go:64] FLAG: --event-burst="100"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283687    2552 flags.go:64] FLAG: --event-qps="50"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283690    2552 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283693    2552 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283696    2552 flags.go:64] FLAG: --eviction-hard=""
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283699    2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283702    2552 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283705    2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283708    2552 flags.go:64] FLAG: --eviction-soft=""
Sep 08 15:44:49.289550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283711    2552 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283714    2552 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283716    2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283719    2552 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283722    2552 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283725    2552 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283728    2552 flags.go:64] FLAG: --feature-gates=""
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283731    2552 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283734    2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283737    2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283741    2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283746    2552 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283750    2552 flags.go:64] FLAG: --help="false"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283753    2552 flags.go:64] FLAG: --hostname-override="ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283756    2552 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283759    2552 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283762    2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283765    2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283769    2552 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283772    2552 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283775    2552 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283777    2552 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283780    2552 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 15:44:49.290207 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283784    2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283787    2552 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283790    2552 flags.go:64] FLAG: --kube-reserved=""
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283793    2552 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283796    2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283799    2552 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283802    2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283805    2552 flags.go:64] FLAG: --lock-file=""
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283807    2552 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283810    2552 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283813    2552 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283818    2552 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283821    2552 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283824    2552 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283826    2552 flags.go:64] FLAG: --logging-format="text"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283830    2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283833    2552 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283836    2552 flags.go:64] FLAG: --manifest-url=""
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283839    2552 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283843    2552 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283847    2552 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283853    2552 flags.go:64] FLAG: --max-pods="110"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283856    2552 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283859    2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283862    2552 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 15:44:49.290875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283865    2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283868    2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283871    2552 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283874    2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283885    2552 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283887    2552 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283890    2552 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283893    2552 flags.go:64] FLAG: --pod-cidr=""
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283897    2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283902    2552 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283905    2552 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283908    2552 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283911    2552 flags.go:64] FLAG: --port="10250"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283914    2552 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283917    2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d607a4b5b8e00e40"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283920    2552 flags.go:64] FLAG: --qos-reserved=""
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283923    2552 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283926    2552 flags.go:64] FLAG: --register-node="true"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283929    2552 flags.go:64] FLAG: --register-with-taints=""
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283933    2552 flags.go:64] FLAG: --registry-burst="10"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283935    2552 flags.go:64] FLAG: --registry-qps="5"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283938    2552 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283941    2552 flags.go:64] FLAG: --reserved-memory=""
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283945    2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283948    2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 15:44:49.291494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283951    2552 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283954    2552 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283958    2552 flags.go:64] FLAG: --runonce="false"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283960    2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283965    2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283969    2552 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283972    2552 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283975    2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283978    2552 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283982    2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283985    2552 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283988    2552 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283991    2552 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283994    2552 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.283997    2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284001    2552 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284007    2552 flags.go:64] FLAG: --system-cgroups=""
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284010    2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284015    2552 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284017    2552 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284020    2552 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284025    2552 flags.go:64] FLAG: --tls-min-version=""
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284028    2552 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284030    2552 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284033    2552 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 15:44:49.292106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284036    2552 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284039    2552 flags.go:64] FLAG: --v="2"
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284043    2552 flags.go:64] FLAG: --version="false"
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284047    2552 flags.go:64] FLAG: --vmodule=""
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284051    2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284054    2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284196    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284200    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284203    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284207    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284210    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284215    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284217    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284220    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284223    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284225    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284228    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284231    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284234    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284237    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284239    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:44:49.292728 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284242    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284245    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284248    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284251    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284254    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284256    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284259    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284262    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284264    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284267    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284269    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284272    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284274    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284277    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284279    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284282    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284284    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284287    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284289    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284292    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:44:49.293230 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284296    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284299    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284303    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284305    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284308    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284322    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284325    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284328    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284330    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284334    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284337    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284339    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284342    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284345    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284347    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284350    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284353    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284357    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284359    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284362    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:44:49.293786 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284364    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284367    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284369    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284372    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284374    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284377    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284380    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284382    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284385    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284388    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284390    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284393    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284397    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284399    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284403    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284406    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284408    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284413    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284416    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:44:49.294271 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284419    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284422    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284424    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284428    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284430    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284433    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284436    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284438    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284441    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284443    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284446    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284449    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284451    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284454    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284456    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284459    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284463    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284466    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284469    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284472    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:44:49.294820 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284475    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284479    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284482    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284485    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.284487    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.284493    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.291766    2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.291781    2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.291800    2552 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.291806    2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291861    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291866    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291869    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291874    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:44:49.295336 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291878    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291881    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291884    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291887    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291890    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291893    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291895    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291898    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291901    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291904    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291908    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291910    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291913    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291916    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291918    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291921    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291924    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291926    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291929    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291932    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:44:49.295715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291934    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291938    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291941    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291945    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291948    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291951    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291954    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291958    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291961    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291964    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291967    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291969    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291972    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291975    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291977    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291980    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291983    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291985    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291988    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:44:49.296201 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291990    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291993    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291996    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.291999    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292001    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292004    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292006    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292009    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292012    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292014    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292017    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292019    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292022    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292024    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292027    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292030    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292033    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292037    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292041    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:44:49.296690 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292045    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292048    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292050    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292053    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292056    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292058    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292061    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292063    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292066    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292068    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292071    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292073    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292075    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292078    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292080    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292083    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292086    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292088    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292091    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292093    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:44:49.297189 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292096    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292098    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292101    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292103    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292106    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292108    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292111    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292114    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292116    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292119    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292123    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292125    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292127    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292130    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292133    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292135    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:44:49.297698 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292138    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.292142    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292268    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292273    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292276    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292280    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292282    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292286    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292290    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292292    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292295    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292298    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292300    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292303    2552 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292306    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:44:49.298091 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292309    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292311    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292330    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292333    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292335    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292338    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292340    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292343    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292345    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292348    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292350    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292353    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292356    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292358    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292361    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292363    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292366    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292368    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292371    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292373    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:44:49.298493 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292376    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292379    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292381    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292385    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292389    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292392    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292395    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292398    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292401    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292403    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292406    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292412    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292415    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292418    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292420    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292423    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292425    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292428    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292430    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:44:49.298999 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292433    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292435    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292437    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292440    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292443    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292446    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292448    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292451    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292453    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292456    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292458    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292460    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292463    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292466    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292468    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292470    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292473    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292475    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292478    2552 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292480    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:44:49.299485 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292482    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292485    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292488    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292490    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292493    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292496    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292498    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292501    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292503    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292505    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292508    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292510    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292513    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292515    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292518    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292520    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292523    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292525    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292528    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292530    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:44:49.299969 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292533    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292535    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292538    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292541    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292545    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292548    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:49.292551    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.292555    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.292654    2552 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.295428    2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.296335    2552 server.go:1019] "Starting client certificate rotation"
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.296431    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.296466    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:44:49.300508 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.298749    2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:44:49.306804 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.306779    2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:44:49.319249 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.319229    2552 log.go:25] "Validated CRI v1 runtime API"
Sep 08 15:44:49.322295 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.322277    2552 log.go:25] "Validated CRI v1 image API"
Sep 08 15:44:49.323161 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.323146    2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 15:44:49.325688 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.325671    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:44:49.326076 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.326056    2552 fs.go:135] Filesystem UUIDs: map[50ecca7d-0877-4e5c-ab95-afeaef6fe08f:/dev/nvme0n1p4 79c9ab72-aa40-4378-84ad-9a9be680e736:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 08 15:44:49.326128 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.326076    2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 15:44:49.331006 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.330886    2552 manager.go:216] Machine: {Timestamp:2026-09-08 15:44:49.329688866 +0000 UTC m=+0.306807683 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108184 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bb6cfc3364ba6dd92dd03950213a6 SystemUUID:ec2bb6cf-c336-4ba6-dd92-dd03950213a6 BootID:816fa372-7c8d-4813-a6d2-0e7945d24597 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:44:fd:7d:d1:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:44:fd:7d:d1:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:02:91:e3:f5:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 15:44:49.331006 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.331000    2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 15:44:49.331126 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.331080    2552 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 15:44:49.333948 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.333923    2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 15:44:49.334084 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.333951    2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-224.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 15:44:49.335409 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.335399    2552 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 15:44:49.335451 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.335411    2552 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 15:44:49.335451 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.335419    2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:44:49.336096 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.336086    2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:44:49.337817 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.337806    2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 15:44:49.341535 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.341522    2552 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:44:49.341644 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.341635    2552 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 15:44:49.341681 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.341664    2552 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 15:44:49.341681 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.341679    2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 15:44:49.341732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.341694    2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 15:44:49.341732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.341703    2552 kubelet.go:398] "Adding apiserver pod source"
Sep 08 15:44:49.341732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.341712    2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 15:44:49.342050 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.342038    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:44:49.342090 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.342056    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:44:49.344132 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.344114    2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 15:44:49.346980 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.346967    2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 15:44:49.347031 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.346997    2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 15:44:49.347210 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347200    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 15:44:49.347243 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347215    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 15:44:49.347243 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347221    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 15:44:49.347243 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347226    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 15:44:49.347243 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347234    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 15:44:49.347243 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347239    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 15:44:49.347701 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347690    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 15:44:49.347740 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347702    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 15:44:49.347740 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347710    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 15:44:49.347740 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347716    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 15:44:49.347740 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347740    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 15:44:49.347843 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.347753    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 15:44:49.348415 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.348404    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 15:44:49.348446 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.348416    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 15:44:49.351506 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.351469    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gj25d"
Sep 08 15:44:49.353847 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.353825    2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-224.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 15:44:49.353847 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.353846    2552 server.go:1268] "Started kubelet"
Sep 08 15:44:49.353991 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.353918    2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 15:44:49.354039 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.353963    2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 15:44:49.354039 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.354029    2552 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 15:44:49.354749 ip-10-0-136-224 systemd[1]: Started Kubernetes Kubelet.
Sep 08 15:44:49.354955 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.354878    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-224.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 15:44:49.354955 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.354882    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 15:44:49.355631 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.355614    2552 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 15:44:49.355862 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.355844    2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 15:44:49.357063 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.356985    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gj25d"
Sep 08 15:44:49.358391 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.358369    2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 15:44:49.360816 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.359903    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-224.ec2.internal.18d362f6d9ebe407  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-224.ec2.internal,UID:ip-10-0-136-224.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-224.ec2.internal,},FirstTimestamp:2026-09-08 15:44:49.353810951 +0000 UTC m=+0.330929754,LastTimestamp:2026-09-08 15:44:49.353810951 +0000 UTC m=+0.330929754,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-224.ec2.internal,}"
Sep 08 15:44:49.361485 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.361463    2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 15:44:49.361485 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.361481    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 15:44:49.361641 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.361566    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:49.362189 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362134    2552 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 15:44:49.362189 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362150    2552 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 15:44:49.362335 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362133    2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 15:44:49.362335 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362295    2552 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 15:44:49.362335 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362302    2552 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 15:44:49.362483 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362460    2552 factory.go:55] Registering systemd factory
Sep 08 15:44:49.362483 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362473    2552 factory.go:223] Registration of the systemd container factory successfully
Sep 08 15:44:49.362747 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362720    2552 factory.go:153] Registering CRI-O factory
Sep 08 15:44:49.362747 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362740    2552 factory.go:223] Registration of the crio container factory successfully
Sep 08 15:44:49.362862 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362796    2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 15:44:49.362862 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362814    2552 factory.go:103] Registering Raw factory
Sep 08 15:44:49.362862 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.362830    2552 manager.go:1196] Started watching for new ooms in manager
Sep 08 15:44:49.363306 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.363293    2552 manager.go:349] Starting recovery of all containers
Sep 08 15:44:49.370041 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.369853    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 15:44:49.370701 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.370677    2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:44:49.374730 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.374707    2552 manager.go:354] Recovery completed
Sep 08 15:44:49.377388 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.377368    2552 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-224.ec2.internal\" not found" node="ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.381027 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.381008    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:44:49.382890 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.382866    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:44:49.382962 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.382897    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:44:49.382962 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.382907    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:44:49.383500 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.383483    2552 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 15:44:49.383500 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.383498    2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 15:44:49.383635 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.383507    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:44:49.383635 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.383517    2552 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:44:49.384208 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.384190    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:44:49.384270 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.384221    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:44:49.384270 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.384231    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:44:49.462697 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.462658    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:49.528072 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.527958    2552 policy_none.go:49] "None policy: Start"
Sep 08 15:44:49.528072 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.527989    2552 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 15:44:49.528072 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.528004    2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 15:44:49.529101 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.529089    2552 policy_none.go:47] "Start"
Sep 08 15:44:49.557005 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.556989    2552 manager.go:337] "Starting Device Plugin manager"
Sep 08 15:44:49.557135 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.557031    2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 15:44:49.557135 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.557042    2552 server.go:87] "Starting device plugin registration server"
Sep 08 15:44:49.557517 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.557501    2552 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 15:44:49.557612 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.557518    2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 15:44:49.558009 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.557956    2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 15:44:49.558111 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.558047    2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 15:44:49.558111 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.558057    2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 15:44:49.558389 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.558367    2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 15:44:49.558472 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.558428    2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:49.613555 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.613521    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 15:44:49.613689 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.613593    2552 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 15:44:49.613689 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.613623    2552 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 15:44:49.613762 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.613728    2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 15:44:49.616001 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.615980    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:44:49.657908 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.657877    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:44:49.658996 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.658963    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:44:49.658996 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.658996    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:44:49.659127 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.659009    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:44:49.659127 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.659035    2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.664929 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.664914    2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.664975 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.664937    2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-224.ec2.internal\": node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:49.685729 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.685697    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:49.714494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.714463    2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal"]
Sep 08 15:44:49.714578 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.714551    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:44:49.715571 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.715551    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:44:49.715685 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.715587    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:44:49.715685 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.715601    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:44:49.717821 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.717805    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:44:49.717939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.717924    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.717985 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.717954    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:44:49.718626 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.718605    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:44:49.718626 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.718611    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:44:49.718737 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.718651    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:44:49.718737 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.718662    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:44:49.718737 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.718630    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:44:49.718737 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.718695    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:44:49.720873 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.720861    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.720914 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.720885    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:44:49.721651 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.721636    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:44:49.721754 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.721662    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:44:49.721754 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.721676    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:44:49.736851 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.736834    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-224.ec2.internal\" not found" node="ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.740386 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.740370    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-224.ec2.internal\" not found" node="ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.764242 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.764210    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96fc930a9b55d927d19f464231ac2a2b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-224.ec2.internal\" (UID: \"96fc930a9b55d927d19f464231ac2a2b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.764242 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.764245    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a9bbefba4567c344b8164f8c9c78584-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal\" (UID: \"4a9bbefba4567c344b8164f8c9c78584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.764423 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.764261    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a9bbefba4567c344b8164f8c9c78584-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal\" (UID: \"4a9bbefba4567c344b8164f8c9c78584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.786152 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.786104    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:49.864636 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.864599    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a9bbefba4567c344b8164f8c9c78584-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal\" (UID: \"4a9bbefba4567c344b8164f8c9c78584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.864636 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.864614    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a9bbefba4567c344b8164f8c9c78584-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal\" (UID: \"4a9bbefba4567c344b8164f8c9c78584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.864791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.864656    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a9bbefba4567c344b8164f8c9c78584-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal\" (UID: \"4a9bbefba4567c344b8164f8c9c78584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.864791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.864683    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96fc930a9b55d927d19f464231ac2a2b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-224.ec2.internal\" (UID: \"96fc930a9b55d927d19f464231ac2a2b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.864791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.864711    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/96fc930a9b55d927d19f464231ac2a2b-config\") pod \"kube-apiserver-proxy-ip-10-0-136-224.ec2.internal\" (UID: \"96fc930a9b55d927d19f464231ac2a2b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.864791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:49.864739    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a9bbefba4567c344b8164f8c9c78584-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal\" (UID: \"4a9bbefba4567c344b8164f8c9c78584\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:49.886699 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.886676    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:49.987171 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:49.987140    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:50.040374 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:50.040299    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:50.044757 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:50.044731    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:51.568089 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:51.568062    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:51.668576 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:51.668536    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:51.730505 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:51.730478    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:44:51.768789 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:51.768763    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:51.776952 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:51.776908    2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 15:44:51.777071 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:51.777050    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 15:44:51.777108 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:51.777075    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 15:44:51.777108 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:51.777080    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 15:44:51.843295 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:51.843244    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 15:39:50 +0000 UTC" deadline="2028-05-01 09:19:16.53619912 +0000 UTC"
Sep 08 15:44:51.843295 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:51.843287    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14417h34m24.692916012s"
Sep 08 15:44:51.869794 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:51.869762    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:51.970328 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:51.970290    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:52.070962 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.070927    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:52.156208 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:52.156175    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9bbefba4567c344b8164f8c9c78584.slice/crio-f401e5984dd69e80506663e194c435640a685763f89c58ca044e5532edb179ae WatchSource:0}: Error finding container f401e5984dd69e80506663e194c435640a685763f89c58ca044e5532edb179ae: Status 404 returned error can't find the container with id f401e5984dd69e80506663e194c435640a685763f89c58ca044e5532edb179ae
Sep 08 15:44:52.160796 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.160781    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:44:52.171792 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.171773    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:52.230689 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:52.230660    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fc930a9b55d927d19f464231ac2a2b.slice/crio-081cef44fae12b3221be87979d96d2999f1e0c6f91b9b32748add0694d075599 WatchSource:0}: Error finding container 081cef44fae12b3221be87979d96d2999f1e0c6f91b9b32748add0694d075599: Status 404 returned error can't find the container with id 081cef44fae12b3221be87979d96d2999f1e0c6f91b9b32748add0694d075599
Sep 08 15:44:52.272593 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.272570    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-224.ec2.internal\" not found"
Sep 08 15:44:52.364235 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.364165    2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:44:52.442976 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.442928    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:52.455767 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.455748    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:44:52.456633 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.456619    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal"
Sep 08 15:44:52.465538 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.465522    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:44:52.823979 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.823894    2552 apiserver.go:52] "Watching apiserver"
Sep 08 15:44:52.832081 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.832055    2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 15:44:52.832403 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.832380    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-dmf8l","openshift-ovn-kubernetes/ovnkube-node-ltprd","kube-system/global-pull-secret-syncer-wqb2w","kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fpvpf","openshift-dns/node-resolver-dkswd","openshift-image-registry/node-ca-kltj9","openshift-multus/multus-additional-cni-plugins-j5xsk","openshift-network-operator/iptables-alerter-42x9f","kube-system/konnectivity-agent-xd4bw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal","openshift-multus/multus-s229p","openshift-multus/network-metrics-daemon-9jph2"]
Sep 08 15:44:52.835076 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.835054    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:52.837061 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.837021    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 15:44:52.837061 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.837059    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-984vf\""
Sep 08 15:44:52.837229 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.837086    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 15:44:52.837229 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.837087    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.838983 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.838956    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 15:44:52.839111 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.839042    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.839187 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.839113    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:44:52.839187 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.839152    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 15:44:52.839306 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.839282    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 15:44:52.839433 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.839416    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 15:44:52.839500 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.839458    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zxh9j\""
Sep 08 15:44:52.841119 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.841099    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:52.841222 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.841172    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:44:52.842318 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.842302    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 15:44:52.843200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.843185    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.845314 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.845297    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:44:52.845415 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.845349    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m4x2n\""
Sep 08 15:44:52.845415 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.845360    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 15:44:52.845526 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.845443    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.847529 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.847513    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 15:44:52.847624 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.847545    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qq84x\""
Sep 08 15:44:52.847624 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.847586    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 15:44:52.847624 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.847599    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:52.849515 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.849497    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:44:52.849613 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.849518    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6xjrr\""
Sep 08 15:44:52.849755 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.849739    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:52.849814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.849757    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 15:44:52.849814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.849771    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 15:44:52.851897 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.851883    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 15:44:52.851993 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.851893    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jq578\""
Sep 08 15:44:52.852186 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852173    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 15:44:52.852322 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852309    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.852853 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852759    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 15:44:52.852853 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852836    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-modprobe-d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.852977 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852897    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-lib-modules\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.852977 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852927    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-tuned\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.852977 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852962    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhv4d\" (UniqueName: \"kubernetes.io/projected/bd80c21f-005f-475b-9a76-e5aabc2f5689-kube-api-access-mhv4d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.852988    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-cni-multus\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853043    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfb6f525-8a24-40f3-841a-6798d886a200-agent-certs\") pod \"konnectivity-agent-xd4bw\" (UID: \"dfb6f525-8a24-40f3-841a-6798d886a200\") " pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:52.853142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853070    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-cnibin\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853100    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-cni-binary-copy\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853125    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-hostroot\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853144    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-conf-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853159    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-sys\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853176    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfb6f525-8a24-40f3-841a-6798d886a200-konnectivity-ca\") pod \"konnectivity-agent-xd4bw\" (UID: \"dfb6f525-8a24-40f3-841a-6798d886a200\") " pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853195    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-system-cni-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853225    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-cni-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853240    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t67f\" (UniqueName: \"kubernetes.io/projected/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-kube-api-access-7t67f\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853261    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysctl-conf\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853289    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-host\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853310    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd80c21f-005f-475b-9a76-e5aabc2f5689-tmp\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853325    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-netns\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853346    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-daemon-config\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853360    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-multus-certs\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853388 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853375    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysctl-d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853387    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/224be9e3-b75d-4315-b6d0-370cf1b6babd-hosts-file\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853434    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-cni-bin\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853453    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-kubelet\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-systemd\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853478    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-run\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853493    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/224be9e3-b75d-4315-b6d0-370cf1b6babd-tmp-dir\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853523    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853555    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-socket-dir-parent\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853582    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-k8s-cni-cncf-io\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853597    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-etc-kubernetes\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853632    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853664    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8gtk\" (UniqueName: \"kubernetes.io/projected/37acb33a-d9a0-484e-94ed-69d21bbb302e-kube-api-access-w8gtk\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853686    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysconfig\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853702    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vplgv\" (UniqueName: \"kubernetes.io/projected/224be9e3-b75d-4315-b6d0-370cf1b6babd-kube-api-access-vplgv\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853746    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80e81107-5b70-4123-87bf-8c0827d277d2-kubelet-config\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853772    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80e81107-5b70-4123-87bf-8c0827d277d2-dbus\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.853824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853796    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-kubernetes\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.854383 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.853816    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-var-lib-kubelet\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.854913    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.854929    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.855005    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wl4zn\""
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.855064    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.855294    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.855363    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.855481    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 15:44:52.855844 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.855485    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 15:44:52.856319 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.856063    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 15:44:52.858385 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.858363    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:44:52.858649 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.858633    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.861165 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.861147    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.863928 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.863912    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 15:44:52.863928 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.863920    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 15:44:52.864092 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.863929    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 15:44:52.864167 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.864146    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 15:44:52.864222 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.864162    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 15:44:52.864222 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.864162    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2p5x4\""
Sep 08 15:44:52.864354 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.864340    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5n4jk\""
Sep 08 15:44:52.880513 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.880496    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5mnl4"
Sep 08 15:44:52.889614 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.889597    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5mnl4"
Sep 08 15:44:52.944360 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.944335    2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 15:44:52.954918 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.954893    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.955054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.954936    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-lib-modules\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.954964    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-tuned\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.954988    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhv4d\" (UniqueName: \"kubernetes.io/projected/bd80c21f-005f-475b-9a76-e5aabc2f5689-kube-api-access-mhv4d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955015    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.955240 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955054    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-serviceca\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:52.955240 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955076    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/224be9e3-b75d-4315-b6d0-370cf1b6babd-tmp-dir\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.955240 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955192    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-lib-modules\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-cni-binary-copy\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955270    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-conf-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955298    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-sys\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955348    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-conf-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955363    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-sys\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955348    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/224be9e3-b75d-4315-b6d0-370cf1b6babd-tmp-dir\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.955610 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955515    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-cni-bin\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.955610 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955543    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9788893-de8d-4fef-bcde-772bd40a940e-ovn-node-metrics-cert\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.955610 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955570    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfb6f525-8a24-40f3-841a-6798d886a200-konnectivity-ca\") pod \"konnectivity-agent-xd4bw\" (UID: \"dfb6f525-8a24-40f3-841a-6798d886a200\") " pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:52.955610 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955595    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-system-cni-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955620    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-cni-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955662    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysctl-conf\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955685    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/224be9e3-b75d-4315-b6d0-370cf1b6babd-hosts-file\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955704    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-system-cni-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955713    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtfvx\" (UniqueName: \"kubernetes.io/projected/5d237297-4138-4c04-9495-cccbbff1e35e-kube-api-access-rtfvx\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955738    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-kubelet\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955762    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ldd4\" (UniqueName: \"kubernetes.io/projected/e9788893-de8d-4fef-bcde-772bd40a940e-kube-api-access-8ldd4\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955786    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-netns\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955810    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysctl-conf\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955809    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-daemon-config\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955842    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-cni-binary-copy\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.955851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955851    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-device-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955860    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-cni-dir\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955888    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-sys-fs\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955896    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/224be9e3-b75d-4315-b6d0-370cf1b6babd-hosts-file\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955919    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-systemd\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955939    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-host\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955959    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-systemd-units\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955979    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-systemd\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956000    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-systemd\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956004    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956061    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-socket-dir-parent\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.956080    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956086    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-etc-kubernetes\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956112    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.956141    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs podName:37acb33a-d9a0-484e-94ed-69d21bbb302e nodeName:}" failed. No retries permitted until 2026-09-08 15:44:53.456112427 +0000 UTC m=+2.952520376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs") pod "network-metrics-daemon-9jph2" (UID: "37acb33a-d9a0-484e-94ed-69d21bbb302e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.955962    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-netns\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956164    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-socket-dir-parent\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.956481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956205    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-etc-kubernetes\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956239    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dfb6f525-8a24-40f3-841a-6798d886a200-konnectivity-ca\") pod \"konnectivity-agent-xd4bw\" (UID: \"dfb6f525-8a24-40f3-841a-6798d886a200\") " pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956238    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956310    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbbr\" (UniqueName: \"kubernetes.io/projected/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-kube-api-access-lqbbr\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956339    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-cni-netd\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956344    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-multus-daemon-config\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956372    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-ovnkube-config\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956399    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-system-cni-dir\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956422    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-env-overrides\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956450    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956474    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956497    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-log-socket\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956519    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-socket-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956546    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-modprobe-d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956569    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df3bbb9f-cad7-43bb-8a5f-d147d6235914-iptables-alerter-script\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956625    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-cni-multus\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.957356 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956649    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfb6f525-8a24-40f3-841a-6798d886a200-agent-certs\") pod \"konnectivity-agent-xd4bw\" (UID: \"dfb6f525-8a24-40f3-841a-6798d886a200\") " pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956667    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-modprobe-d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956674    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-cnibin\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956693    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-cni-multus\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956738    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-cnibin\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956771    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-hostroot\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956801    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd80c21f-005f-475b-9a76-e5aabc2f5689-tmp\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956827    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-etc-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956853    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956881    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-slash\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956873    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-hostroot\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956909    2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956926    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t67f\" (UniqueName: \"kubernetes.io/projected/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-kube-api-access-7t67f\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956971    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-host\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.956996    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957020    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-run-netns\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957080    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6b46\" (UniqueName: \"kubernetes.io/projected/6b67e8db-d511-4e40-8974-d97cfdce85ed-kube-api-access-v6b46\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957106    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-multus-certs\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957131    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysctl-d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957158    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-ovnkube-script-lib\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957288    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysctl-d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957375    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-registration-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957407    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957425    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-host\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957430    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-cni-bin\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957454    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-kubelet\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957476    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-run\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957487    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-cni-bin\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957498    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df3bbb9f-cad7-43bb-8a5f-d147d6235914-host-slash\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957500    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-var-lib-kubelet\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957537    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-run\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957540    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-k8s-cni-cncf-io\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957557    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-k8s-cni-cncf-io\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957567    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-cnibin\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957565    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-host-run-multus-certs\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.958782 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957613    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-var-lib-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957637    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-ovn\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957669    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957690    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8gtk\" (UniqueName: \"kubernetes.io/projected/37acb33a-d9a0-484e-94ed-69d21bbb302e-kube-api-access-w8gtk\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957709    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysconfig\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957728    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vplgv\" (UniqueName: \"kubernetes.io/projected/224be9e3-b75d-4315-b6d0-370cf1b6babd-kube-api-access-vplgv\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957747    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-node-log\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957892    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-run-ovn-kubernetes\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957925    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80e81107-5b70-4123-87bf-8c0827d277d2-kubelet-config\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957951    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80e81107-5b70-4123-87bf-8c0827d277d2-dbus\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957973    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-kubernetes\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957987    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-var-lib-kubelet\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.957993    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80e81107-5b70-4123-87bf-8c0827d277d2-kubelet-config\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.958002    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrjf7\" (UniqueName: \"kubernetes.io/projected/df3bbb9f-cad7-43bb-8a5f-d147d6235914-kube-api-access-rrjf7\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.958050    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-kubernetes\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.958095    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-var-lib-kubelet\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.958095    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80e81107-5b70-4123-87bf-8c0827d277d2-dbus\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:52.959935 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.958431    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:52.961087 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:52.958511    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret podName:80e81107-5b70-4123-87bf-8c0827d277d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:44:53.458493621 +0000 UTC m=+2.954901570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret") pod "global-pull-secret-syncer-wqb2w" (UID: "80e81107-5b70-4123-87bf-8c0827d277d2") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:52.961087 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.958584    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-sysconfig\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.961087 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.960873    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd80c21f-005f-475b-9a76-e5aabc2f5689-tmp\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.961087 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.960925    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd80c21f-005f-475b-9a76-e5aabc2f5689-etc-tuned\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:52.961347 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.961326    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dfb6f525-8a24-40f3-841a-6798d886a200-agent-certs\") pod \"konnectivity-agent-xd4bw\" (UID: \"dfb6f525-8a24-40f3-841a-6798d886a200\") " pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:52.968305 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.968278    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vplgv\" (UniqueName: \"kubernetes.io/projected/224be9e3-b75d-4315-b6d0-370cf1b6babd-kube-api-access-vplgv\") pod \"node-resolver-dkswd\" (UID: \"224be9e3-b75d-4315-b6d0-370cf1b6babd\") " pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:52.968422 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.968401    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t67f\" (UniqueName: \"kubernetes.io/projected/1aa25f67-3646-4a1c-8b1b-d4117e8e4898-kube-api-access-7t67f\") pod \"multus-s229p\" (UID: \"1aa25f67-3646-4a1c-8b1b-d4117e8e4898\") " pod="openshift-multus/multus-s229p"
Sep 08 15:44:52.968644 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.968611    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8gtk\" (UniqueName: \"kubernetes.io/projected/37acb33a-d9a0-484e-94ed-69d21bbb302e-kube-api-access-w8gtk\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:52.969197 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:52.969171    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhv4d\" (UniqueName: \"kubernetes.io/projected/bd80c21f-005f-475b-9a76-e5aabc2f5689-kube-api-access-mhv4d\") pod \"tuned-fpvpf\" (UID: \"bd80c21f-005f-475b-9a76-e5aabc2f5689\") " pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:53.017999 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.017964    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:44:53.028750 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.028729    2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:44:53.058462 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058429    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.058623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058471    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.058623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058498    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbbr\" (UniqueName: \"kubernetes.io/projected/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-kube-api-access-lqbbr\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:53.058623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058521    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-cni-netd\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058544    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-ovnkube-config\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058566    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-system-cni-dir\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.058623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058587    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-env-overrides\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058625    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058650    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058673    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-log-socket\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058699    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-socket-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058717    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058741    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df3bbb9f-cad7-43bb-8a5f-d147d6235914-iptables-alerter-script\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058781    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-cni-netd\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058787    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-etc-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058826    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058832    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-etc-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058853    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-slash\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058881    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058906    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-run-netns\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.058939 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058930    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6b46\" (UniqueName: \"kubernetes.io/projected/6b67e8db-d511-4e40-8974-d97cfdce85ed-kube-api-access-v6b46\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058960    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-ovnkube-script-lib\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.058984    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-registration-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059007    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059052    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df3bbb9f-cad7-43bb-8a5f-d147d6235914-host-slash\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059077    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-cnibin\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059093    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059103    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-var-lib-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059136    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-ovn\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059142    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-var-lib-openvswitch\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059180    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-node-log\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059203    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-run-ovn-kubernetes\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059220    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-env-overrides\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059233    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrjf7\" (UniqueName: \"kubernetes.io/projected/df3bbb9f-cad7-43bb-8a5f-d147d6235914-kube-api-access-rrjf7\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059281    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059313    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059339    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-serviceca\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:53.059658 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059370    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-cni-bin\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059402    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9788893-de8d-4fef-bcde-772bd40a940e-ovn-node-metrics-cert\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059401    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-ovnkube-config\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059434    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtfvx\" (UniqueName: \"kubernetes.io/projected/5d237297-4138-4c04-9495-cccbbff1e35e-kube-api-access-rtfvx\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059458    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-kubelet\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059461    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-system-cni-dir\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059493    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ldd4\" (UniqueName: \"kubernetes.io/projected/e9788893-de8d-4fef-bcde-772bd40a940e-kube-api-access-8ldd4\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059509    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-kubelet\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059525    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-device-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059549    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-log-socket\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059605    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-sys-fs\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059636    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-ovn\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059661    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059686    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-node-log\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059722    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-run-ovn-kubernetes\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059760    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-cni-bin\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059787    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-device-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.060464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059837    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059909    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-socket-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060063    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060066    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df3bbb9f-cad7-43bb-8a5f-d147d6235914-iptables-alerter-script\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060127    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-run-netns\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060238    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060386    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e9788893-de8d-4fef-bcde-772bd40a940e-ovnkube-script-lib\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060426    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060443    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df3bbb9f-cad7-43bb-8a5f-d147d6235914-host-slash\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060484    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-registration-dir\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060492    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d237297-4138-4c04-9495-cccbbff1e35e-cnibin\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.059550    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b67e8db-d511-4e40-8974-d97cfdce85ed-sys-fs\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060528    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-host\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060553    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-systemd-units\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060576    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-systemd\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060605    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-serviceca\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060635    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-systemd-units\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.061279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060679    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-run-systemd\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.062080 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060681    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9788893-de8d-4fef-bcde-772bd40a940e-host-slash\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.062080 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.060787    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-host\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:53.062080 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.061047    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d237297-4138-4c04-9495-cccbbff1e35e-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.063159 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.063096    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9788893-de8d-4fef-bcde-772bd40a940e-ovn-node-metrics-cert\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.067759 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.067734    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrjf7\" (UniqueName: \"kubernetes.io/projected/df3bbb9f-cad7-43bb-8a5f-d147d6235914-kube-api-access-rrjf7\") pod \"iptables-alerter-42x9f\" (UID: \"df3bbb9f-cad7-43bb-8a5f-d147d6235914\") " pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:53.071264 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.071239    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:44:53.071264 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.071264    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:44:53.071425 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.071284    2552 projected.go:196] Error preparing data for projected volume kube-api-access-f72d9 for pod openshift-network-diagnostics/network-check-target-dmf8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:53.071425 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.071350    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9 podName:0ab39284-a215-4381-b79a-faf6edfb5cb0 nodeName:}" failed. No retries permitted until 2026-09-08 15:44:53.571331558 +0000 UTC m=+3.067739523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f72d9" (UniqueName: "kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9") pod "network-check-target-dmf8l" (UID: "0ab39284-a215-4381-b79a-faf6edfb5cb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:53.073689 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.073670    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ldd4\" (UniqueName: \"kubernetes.io/projected/e9788893-de8d-4fef-bcde-772bd40a940e-kube-api-access-8ldd4\") pod \"ovnkube-node-ltprd\" (UID: \"e9788893-de8d-4fef-bcde-772bd40a940e\") " pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.074279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.074218    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbbr\" (UniqueName: \"kubernetes.io/projected/4a606414-2982-4f7c-9cfa-0fbc3e7d84ad-kube-api-access-lqbbr\") pod \"node-ca-kltj9\" (UID: \"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad\") " pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:53.074677 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.074643    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6b46\" (UniqueName: \"kubernetes.io/projected/6b67e8db-d511-4e40-8974-d97cfdce85ed-kube-api-access-v6b46\") pod \"aws-ebs-csi-driver-node-thj8d\" (UID: \"6b67e8db-d511-4e40-8974-d97cfdce85ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.075018 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.074992    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtfvx\" (UniqueName: \"kubernetes.io/projected/5d237297-4138-4c04-9495-cccbbff1e35e-kube-api-access-rtfvx\") pod \"multus-additional-cni-plugins-j5xsk\" (UID: \"5d237297-4138-4c04-9495-cccbbff1e35e\") " pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.099614 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.099559    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal" event={"ID":"96fc930a9b55d927d19f464231ac2a2b","Type":"ContainerStarted","Data":"081cef44fae12b3221be87979d96d2999f1e0c6f91b9b32748add0694d075599"}
Sep 08 15:44:53.100876 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.100848    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal" event={"ID":"4a9bbefba4567c344b8164f8c9c78584","Type":"ContainerStarted","Data":"f401e5984dd69e80506663e194c435640a685763f89c58ca044e5532edb179ae"}
Sep 08 15:44:53.144081 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.144053    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:44:53.150398 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.150367    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s229p"
Sep 08 15:44:53.158891 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.158688    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb6f525_8a24_40f3_841a_6798d886a200.slice/crio-27198bb0172de0dad9dc87fe4cb21832adf33857976484c4ef0ea65cfc9e97b7 WatchSource:0}: Error finding container 27198bb0172de0dad9dc87fe4cb21832adf33857976484c4ef0ea65cfc9e97b7: Status 404 returned error can't find the container with id 27198bb0172de0dad9dc87fe4cb21832adf33857976484c4ef0ea65cfc9e97b7
Sep 08 15:44:53.159173 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.158986    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fpvpf"
Sep 08 15:44:53.161395 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.161363    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa25f67_3646_4a1c_8b1b_d4117e8e4898.slice/crio-b5adf37ed3c114dddbc3bf9cf035d6af457e4277c4826ed5d3cc8d8955135f9e WatchSource:0}: Error finding container b5adf37ed3c114dddbc3bf9cf035d6af457e4277c4826ed5d3cc8d8955135f9e: Status 404 returned error can't find the container with id b5adf37ed3c114dddbc3bf9cf035d6af457e4277c4826ed5d3cc8d8955135f9e
Sep 08 15:44:53.162793 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.162678    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dkswd"
Sep 08 15:44:53.169020 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.169002    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-42x9f"
Sep 08 15:44:53.170865 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.170835    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd80c21f_005f_475b_9a76_e5aabc2f5689.slice/crio-cf0a6b46f3a480faf62b26fc884fc91c4197500c878d0ba6d96cbd7df1f40f60 WatchSource:0}: Error finding container cf0a6b46f3a480faf62b26fc884fc91c4197500c878d0ba6d96cbd7df1f40f60: Status 404 returned error can't find the container with id cf0a6b46f3a480faf62b26fc884fc91c4197500c878d0ba6d96cbd7df1f40f60
Sep 08 15:44:53.174355 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.174336    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kltj9"
Sep 08 15:44:53.174768 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.174742    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod224be9e3_b75d_4315_b6d0_370cf1b6babd.slice/crio-d1e21b4b9facd6ba867544daf091eeced5d3859d984917dca73e77a6438587a4 WatchSource:0}: Error finding container d1e21b4b9facd6ba867544daf091eeced5d3859d984917dca73e77a6438587a4: Status 404 returned error can't find the container with id d1e21b4b9facd6ba867544daf091eeced5d3859d984917dca73e77a6438587a4
Sep 08 15:44:53.179664 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.179635    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:44:53.182143 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.182114    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3bbb9f_cad7_43bb_8a5f_d147d6235914.slice/crio-7624081885bf58ef8bba02f1aebf838f9ae5ae53dddcffb9de1d6a7c5377b8af WatchSource:0}: Error finding container 7624081885bf58ef8bba02f1aebf838f9ae5ae53dddcffb9de1d6a7c5377b8af: Status 404 returned error can't find the container with id 7624081885bf58ef8bba02f1aebf838f9ae5ae53dddcffb9de1d6a7c5377b8af
Sep 08 15:44:53.185193 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.185001    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d"
Sep 08 15:44:53.187607 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.187523    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a606414_2982_4f7c_9cfa_0fbc3e7d84ad.slice/crio-f7b95e832a70e640b8c63beb5beec23823284b9534db64d19d474d18d049d99d WatchSource:0}: Error finding container f7b95e832a70e640b8c63beb5beec23823284b9534db64d19d474d18d049d99d: Status 404 returned error can't find the container with id f7b95e832a70e640b8c63beb5beec23823284b9534db64d19d474d18d049d99d
Sep 08 15:44:53.189519 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.189320    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5xsk"
Sep 08 15:44:53.192901 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.192877    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9788893_de8d_4fef_bcde_772bd40a940e.slice/crio-b5f751181ece62e38edb6a8abb48f317558c85fe1ec637299018738b6c73c7c2 WatchSource:0}: Error finding container b5f751181ece62e38edb6a8abb48f317558c85fe1ec637299018738b6c73c7c2: Status 404 returned error can't find the container with id b5f751181ece62e38edb6a8abb48f317558c85fe1ec637299018738b6c73c7c2
Sep 08 15:44:53.200445 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.200213    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b67e8db_d511_4e40_8974_d97cfdce85ed.slice/crio-6373f0b828f889e091ee07e4ffdc4f73c8371eb0d21e693f29fb4630647c4418 WatchSource:0}: Error finding container 6373f0b828f889e091ee07e4ffdc4f73c8371eb0d21e693f29fb4630647c4418: Status 404 returned error can't find the container with id 6373f0b828f889e091ee07e4ffdc4f73c8371eb0d21e693f29fb4630647c4418
Sep 08 15:44:53.203777 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:44:53.203404    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d237297_4138_4c04_9495_cccbbff1e35e.slice/crio-c2dfd7070c0446f94470cae19a3ec03f9e83441b2fe4f262434f553fdcf8c0c8 WatchSource:0}: Error finding container c2dfd7070c0446f94470cae19a3ec03f9e83441b2fe4f262434f553fdcf8c0c8: Status 404 returned error can't find the container with id c2dfd7070c0446f94470cae19a3ec03f9e83441b2fe4f262434f553fdcf8c0c8
Sep 08 15:44:53.463687 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.463590    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:53.463687 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.463648    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:53.463897 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.463770    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:53.463897 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.463773    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:53.463897 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.463827    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret podName:80e81107-5b70-4123-87bf-8c0827d277d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:44:54.463814541 +0000 UTC m=+3.960222491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret") pod "global-pull-secret-syncer-wqb2w" (UID: "80e81107-5b70-4123-87bf-8c0827d277d2") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:53.463897 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.463847    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs podName:37acb33a-d9a0-484e-94ed-69d21bbb302e nodeName:}" failed. No retries permitted until 2026-09-08 15:44:54.463837027 +0000 UTC m=+3.960244976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs") pod "network-metrics-daemon-9jph2" (UID: "37acb33a-d9a0-484e-94ed-69d21bbb302e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:53.664755 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.664720    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:53.664939 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.664918    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:44:53.665002 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.664946    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:44:53.665002 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.664962    2552 projected.go:196] Error preparing data for projected volume kube-api-access-f72d9 for pod openshift-network-diagnostics/network-check-target-dmf8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:53.665123 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:53.665036    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9 podName:0ab39284-a215-4381-b79a-faf6edfb5cb0 nodeName:}" failed. No retries permitted until 2026-09-08 15:44:54.665008103 +0000 UTC m=+4.161416048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f72d9" (UniqueName: "kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9") pod "network-check-target-dmf8l" (UID: "0ab39284-a215-4381-b79a-faf6edfb5cb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:53.893046 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.892985    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:39:52 +0000 UTC" deadline="2028-05-23 19:22:18.99717348 +0000 UTC"
Sep 08 15:44:53.893046 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:53.893036    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14955h37m25.104153777s"
Sep 08 15:44:54.095086 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.095060    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:54.095203 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.095158    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:44:54.103909 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.103880    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s229p" event={"ID":"1aa25f67-3646-4a1c-8b1b-d4117e8e4898","Type":"ContainerStarted","Data":"b5adf37ed3c114dddbc3bf9cf035d6af457e4277c4826ed5d3cc8d8955135f9e"}
Sep 08 15:44:54.105697 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.105433    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerStarted","Data":"c2dfd7070c0446f94470cae19a3ec03f9e83441b2fe4f262434f553fdcf8c0c8"}
Sep 08 15:44:54.106731 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.106707    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kltj9" event={"ID":"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad","Type":"ContainerStarted","Data":"f7b95e832a70e640b8c63beb5beec23823284b9534db64d19d474d18d049d99d"}
Sep 08 15:44:54.108208 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.108184    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xd4bw" event={"ID":"dfb6f525-8a24-40f3-841a-6798d886a200","Type":"ContainerStarted","Data":"27198bb0172de0dad9dc87fe4cb21832adf33857976484c4ef0ea65cfc9e97b7"}
Sep 08 15:44:54.109831 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.109807    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d" event={"ID":"6b67e8db-d511-4e40-8974-d97cfdce85ed","Type":"ContainerStarted","Data":"6373f0b828f889e091ee07e4ffdc4f73c8371eb0d21e693f29fb4630647c4418"}
Sep 08 15:44:54.111387 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.111361    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"b5f751181ece62e38edb6a8abb48f317558c85fe1ec637299018738b6c73c7c2"}
Sep 08 15:44:54.112839 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.112815    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-42x9f" event={"ID":"df3bbb9f-cad7-43bb-8a5f-d147d6235914","Type":"ContainerStarted","Data":"7624081885bf58ef8bba02f1aebf838f9ae5ae53dddcffb9de1d6a7c5377b8af"}
Sep 08 15:44:54.114317 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.114294    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dkswd" event={"ID":"224be9e3-b75d-4315-b6d0-370cf1b6babd","Type":"ContainerStarted","Data":"d1e21b4b9facd6ba867544daf091eeced5d3859d984917dca73e77a6438587a4"}
Sep 08 15:44:54.115796 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.115772    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fpvpf" event={"ID":"bd80c21f-005f-475b-9a76-e5aabc2f5689","Type":"ContainerStarted","Data":"cf0a6b46f3a480faf62b26fc884fc91c4197500c878d0ba6d96cbd7df1f40f60"}
Sep 08 15:44:54.391242 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.390902    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:44:54.471976 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.471890    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:54.472163 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.471975    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:54.472163 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.472125    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:54.472279 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.472183    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:54.472279 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.472193    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret podName:80e81107-5b70-4123-87bf-8c0827d277d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:44:56.472173773 +0000 UTC m=+5.968581722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret") pod "global-pull-secret-syncer-wqb2w" (UID: "80e81107-5b70-4123-87bf-8c0827d277d2") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:54.472279 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.472265    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs podName:37acb33a-d9a0-484e-94ed-69d21bbb302e nodeName:}" failed. No retries permitted until 2026-09-08 15:44:56.472246349 +0000 UTC m=+5.968654307 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs") pod "network-metrics-daemon-9jph2" (UID: "37acb33a-d9a0-484e-94ed-69d21bbb302e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:54.674558 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.673925    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:54.674558 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.674091    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:44:54.674558 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.674119    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:44:54.674558 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.674134    2552 projected.go:196] Error preparing data for projected volume kube-api-access-f72d9 for pod openshift-network-diagnostics/network-check-target-dmf8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:54.674558 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:54.674218    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9 podName:0ab39284-a215-4381-b79a-faf6edfb5cb0 nodeName:}" failed. No retries permitted until 2026-09-08 15:44:56.674197371 +0000 UTC m=+6.170605332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f72d9" (UniqueName: "kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9") pod "network-check-target-dmf8l" (UID: "0ab39284-a215-4381-b79a-faf6edfb5cb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:54.893727 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.893636    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:39:52 +0000 UTC" deadline="2028-04-15 19:36:18.845008589 +0000 UTC"
Sep 08 15:44:54.893727 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:54.893695    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14043h51m23.951318068s"
Sep 08 15:44:55.098566 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:55.097813    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:55.098566 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:55.097948    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:44:55.098566 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:55.098366    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:55.098566 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:55.098511    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:44:55.127208 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:55.126315    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal" event={"ID":"96fc930a9b55d927d19f464231ac2a2b","Type":"ContainerStarted","Data":"3b5dfea458c71d23e9687ad9ce42724cee13c1f4f792b2afac1168ce039e89c6"}
Sep 08 15:44:56.095208 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:56.095176    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:56.095699 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.095319    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:44:56.488548 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:56.488510    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:56.488733 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:56.488599    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:56.488799 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.488748    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:56.488856 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.488805    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret podName:80e81107-5b70-4123-87bf-8c0827d277d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:00.488786288 +0000 UTC m=+9.985194253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret") pod "global-pull-secret-syncer-wqb2w" (UID: "80e81107-5b70-4123-87bf-8c0827d277d2") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:44:56.489185 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.489161    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:56.489289 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.489223    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs podName:37acb33a-d9a0-484e-94ed-69d21bbb302e nodeName:}" failed. No retries permitted until 2026-09-08 15:45:00.489206772 +0000 UTC m=+9.985614722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs") pod "network-metrics-daemon-9jph2" (UID: "37acb33a-d9a0-484e-94ed-69d21bbb302e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:44:56.690614 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:56.690576    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:56.690840 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.690738    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:44:56.690840 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.690764    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:44:56.690840 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.690777    2552 projected.go:196] Error preparing data for projected volume kube-api-access-f72d9 for pod openshift-network-diagnostics/network-check-target-dmf8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:56.691010 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:56.690834    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9 podName:0ab39284-a215-4381-b79a-faf6edfb5cb0 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:00.690817609 +0000 UTC m=+10.187225560 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f72d9" (UniqueName: "kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9") pod "network-check-target-dmf8l" (UID: "0ab39284-a215-4381-b79a-faf6edfb5cb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:44:57.095003 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:57.094975    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:57.095213 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:57.095114    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:44:57.097741 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:57.094937    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:57.098143 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:57.097841    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:44:58.094802 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:58.094765    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:44:58.094993 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:58.094892    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:44:59.095275 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:59.094905    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:44:59.095275 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:59.095066    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:44:59.096048 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:44:59.095346    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:44:59.096048 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:44:59.095450    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:00.094931 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:00.094853    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:00.095170 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.094991    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:00.525273 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:00.525229    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:00.525688 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:00.525300    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:00.525688 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.525370    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:45:00.525688 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.525395    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:45:00.525688 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.525434    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret podName:80e81107-5b70-4123-87bf-8c0827d277d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:08.525416263 +0000 UTC m=+18.021824211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret") pod "global-pull-secret-syncer-wqb2w" (UID: "80e81107-5b70-4123-87bf-8c0827d277d2") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:45:00.525688 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.525453    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs podName:37acb33a-d9a0-484e-94ed-69d21bbb302e nodeName:}" failed. No retries permitted until 2026-09-08 15:45:08.525443849 +0000 UTC m=+18.021851794 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs") pod "network-metrics-daemon-9jph2" (UID: "37acb33a-d9a0-484e-94ed-69d21bbb302e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:45:00.727664 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:00.727621    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:00.727846 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.727776    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:45:00.727846 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.727796    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:45:00.727846 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.727809    2552 projected.go:196] Error preparing data for projected volume kube-api-access-f72d9 for pod openshift-network-diagnostics/network-check-target-dmf8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:45:00.728014 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:00.727865    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9 podName:0ab39284-a215-4381-b79a-faf6edfb5cb0 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:08.727848934 +0000 UTC m=+18.224256894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f72d9" (UniqueName: "kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9") pod "network-check-target-dmf8l" (UID: "0ab39284-a215-4381-b79a-faf6edfb5cb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:45:01.095481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:01.095445    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:01.095668 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:01.095570    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:01.095668 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:01.095601    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:01.095795 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:01.095704    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:02.094611 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:02.094572    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:02.095124 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:02.094715    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:03.098533 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:03.098501    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:03.099012 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:03.098630    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:03.099012 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:03.098685    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:03.099012 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:03.098824    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:04.095312 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:04.095273    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:04.095505 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:04.095425    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:05.095510 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:05.095472    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:05.096084 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:05.095515    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:05.096084 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:05.095617    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:05.096084 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:05.095718    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:06.094874 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:06.094829    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:06.095087 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:06.094955    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:07.094723 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:07.094689    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:07.095144 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:07.094816    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:07.095144 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:07.094851    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:07.095144 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:07.094923    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:08.094827 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:08.094792    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:08.095284 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.094917    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:08.590585 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:08.590536    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:08.590773 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:08.590609    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:08.590773 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.590705    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:45:08.590773 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.590721    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:45:08.590938 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.590778    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs podName:37acb33a-d9a0-484e-94ed-69d21bbb302e nodeName:}" failed. No retries permitted until 2026-09-08 15:45:24.590758333 +0000 UTC m=+34.087166297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs") pod "network-metrics-daemon-9jph2" (UID: "37acb33a-d9a0-484e-94ed-69d21bbb302e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:45:08.590938 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.590797    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret podName:80e81107-5b70-4123-87bf-8c0827d277d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:24.59078764 +0000 UTC m=+34.087195599 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret") pod "global-pull-secret-syncer-wqb2w" (UID: "80e81107-5b70-4123-87bf-8c0827d277d2") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:45:08.792540 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:08.792503    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:08.792706 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.792649    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:45:08.792706 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.792668    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:45:08.792706 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.792679    2552 projected.go:196] Error preparing data for projected volume kube-api-access-f72d9 for pod openshift-network-diagnostics/network-check-target-dmf8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:45:08.792877 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:08.792745    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9 podName:0ab39284-a215-4381-b79a-faf6edfb5cb0 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:24.792726947 +0000 UTC m=+34.289134895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f72d9" (UniqueName: "kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9") pod "network-check-target-dmf8l" (UID: "0ab39284-a215-4381-b79a-faf6edfb5cb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:45:09.095254 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:09.095182    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:09.095254 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:09.095219    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:09.095777 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:09.095346    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:09.095777 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:09.095436    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:10.095561 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:10.095527    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:10.096018 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:10.095653    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:11.097622 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:11.097551    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:11.097622 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:11.097565    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:11.098062 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:11.097658    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:11.098062 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:11.097823    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:12.095560 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.095248    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:12.095693 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:12.095673    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:12.161529 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.161495    2552 generic.go:356] "Generic (PLEG): container finished" podID="4a9bbefba4567c344b8164f8c9c78584" containerID="b6a8c014bc33722179c446d275596929bc1570bebb372fdb10c5f77847cb4038" exitCode=0
Sep 08 15:45:12.162268 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.161591    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal" event={"ID":"4a9bbefba4567c344b8164f8c9c78584","Type":"ContainerDied","Data":"b6a8c014bc33722179c446d275596929bc1570bebb372fdb10c5f77847cb4038"}
Sep 08 15:45:12.163294 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.163267    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d" event={"ID":"6b67e8db-d511-4e40-8974-d97cfdce85ed","Type":"ContainerStarted","Data":"991e279dbc777bd19b0c8d1891228f6b53d9ce733bc0ef05c8f9c014cb7be751"}
Sep 08 15:45:12.166701 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.166678    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"cd9ad5fe4bd82962188f73df9924222a36890ecc2c7db1a0e828a4162fef517f"}
Sep 08 15:45:12.166793 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.166774    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"2cebd743e2a7d50bb74c7f13a1ab12fd9673d301cdee2c732c325eb7606cd2ed"}
Sep 08 15:45:12.166846 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.166829    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"3e516cd7d2a7c2560b78d700473e036c73dc5d86799d693e4353b19263093ebd"}
Sep 08 15:45:12.166894 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.166881    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"55f863fda3e3860eaa5f861652c6c5845ba42de81c0144f27911d29c17c04ccb"}
Sep 08 15:45:12.166962 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.166932    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"7809538dd81ed5965ef82868a5e07fb2842327b7bc9e42f9e20801899369609f"}
Sep 08 15:45:12.167016 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.166985    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"768aa443a52ae16de18aea55e0a91ffb28901f0c84d3e3acfe465c544120815d"}
Sep 08 15:45:12.168249 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.168222    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dkswd" event={"ID":"224be9e3-b75d-4315-b6d0-370cf1b6babd","Type":"ContainerStarted","Data":"de9d737f220926d03bbcb2aa1691329c36538030d49037797c9f89cc13d16a9d"}
Sep 08 15:45:12.169827 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.169806    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fpvpf" event={"ID":"bd80c21f-005f-475b-9a76-e5aabc2f5689","Type":"ContainerStarted","Data":"20c477bf20282638f4019b72a8b0fd882911b2e62e1f9b28174d8c1a486c89aa"}
Sep 08 15:45:12.171384 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.171361    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s229p" event={"ID":"1aa25f67-3646-4a1c-8b1b-d4117e8e4898","Type":"ContainerStarted","Data":"b8236431689ccb6fb7ea954ab8909fc4fa6c103fadcbc3943e34e1fbe07bea51"}
Sep 08 15:45:12.172793 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.172768    2552 generic.go:356] "Generic (PLEG): container finished" podID="5d237297-4138-4c04-9495-cccbbff1e35e" containerID="e60752ba708d25ab07fa116e4d6e3d2835638cc274124f15d03aa1bb56a45c58" exitCode=0
Sep 08 15:45:12.172858 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.172836    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerDied","Data":"e60752ba708d25ab07fa116e4d6e3d2835638cc274124f15d03aa1bb56a45c58"}
Sep 08 15:45:12.174524 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.174500    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kltj9" event={"ID":"4a606414-2982-4f7c-9cfa-0fbc3e7d84ad","Type":"ContainerStarted","Data":"cc7a9839642bbc59aeb975eb101138b61f0452d68ac2cea7443fd4333fb560c7"}
Sep 08 15:45:12.175905 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.175880    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xd4bw" event={"ID":"dfb6f525-8a24-40f3-841a-6798d886a200","Type":"ContainerStarted","Data":"d2f8256f3382fbd3871affd0a4fced7c767d0de49e6c2df893cf63be1cd9aa3b"}
Sep 08 15:45:12.176062 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.176008    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-224.ec2.internal" podStartSLOduration=20.175994974 podStartE2EDuration="20.175994974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:44:55.150729061 +0000 UTC m=+4.647137029" watchObservedRunningTime="2026-09-08 15:45:12.175994974 +0000 UTC m=+21.672402941"
Sep 08 15:45:12.225451 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.225389    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fpvpf" podStartSLOduration=3.067487384 podStartE2EDuration="21.225368091s" totalImagesPullingTime="18.157880707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:12.209589921 +0000 UTC m=+21.705997889" watchObservedRunningTime="2026-09-08 15:45:12.225368091 +0000 UTC m=+21.721776049"
Sep 08 15:45:12.244071 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.244003    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dkswd" podStartSLOduration=3.09247767 podStartE2EDuration="21.243986209s" totalImagesPullingTime="18.151508539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:12.225525922 +0000 UTC m=+21.721933884" watchObservedRunningTime="2026-09-08 15:45:12.243986209 +0000 UTC m=+21.740394169"
Sep 08 15:45:12.244311 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.244285    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xd4bw" podStartSLOduration=2.886711782 podStartE2EDuration="21.244275379s" totalImagesPullingTime="18.357563597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:12.244101241 +0000 UTC m=+21.740509206" watchObservedRunningTime="2026-09-08 15:45:12.244275379 +0000 UTC m=+21.740683349"
Sep 08 15:45:12.275967 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.275926    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kltj9" podStartSLOduration=7.558068173 podStartE2EDuration="21.275911006s" totalImagesPullingTime="13.717842833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:12.275757803 +0000 UTC m=+21.772165771" watchObservedRunningTime="2026-09-08 15:45:12.275911006 +0000 UTC m=+21.772318972"
Sep 08 15:45:12.276281 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:12.276253    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s229p" podStartSLOduration=2.837972324 podStartE2EDuration="21.276243679s" totalImagesPullingTime="18.438271355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:12.260792565 +0000 UTC m=+21.757200539" watchObservedRunningTime="2026-09-08 15:45:12.276243679 +0000 UTC m=+21.772651670"
Sep 08 15:45:13.097430 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:13.097404    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:13.097430 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:13.097426    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:13.097698 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:13.097507    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:13.097698 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:13.097620    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:13.179801 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:13.179761    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal" event={"ID":"4a9bbefba4567c344b8164f8c9c78584","Type":"ContainerStarted","Data":"825bd2f0fa5495203653df9c5cd308835a16fe037f6a91b65a5570e54fb7af1d"}
Sep 08 15:45:13.181348 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:13.181317    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-42x9f" event={"ID":"df3bbb9f-cad7-43bb-8a5f-d147d6235914","Type":"ContainerStarted","Data":"c8c7b8702ccb75ba0679db836775dfd1b8263c785ede85139872fee926426611"}
Sep 08 15:45:13.195168 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:13.195126    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-224.ec2.internal" podStartSLOduration=21.195112706 podStartE2EDuration="21.195112706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:13.194636003 +0000 UTC m=+22.691043971" watchObservedRunningTime="2026-09-08 15:45:13.195112706 +0000 UTC m=+22.691520675"
Sep 08 15:45:13.772856 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:13.772648    2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 15:45:14.051114 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.050942    2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T15:45:13.772854371Z","UUID":"98911b0a-f6fd-4b9a-af9e-57c04fc906b0","Handler":null,"Name":"","Endpoint":""}
Sep 08 15:45:14.052391 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.052373    2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 15:45:14.052506 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.052406    2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 15:45:14.094701 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.094671    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:14.094869 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:14.094787    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:14.185100 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.185070    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d" event={"ID":"6b67e8db-d511-4e40-8974-d97cfdce85ed","Type":"ContainerStarted","Data":"5f67a7e47fc5f0d178bc7922f3eb9d91b55c601ea0214f2def854fe585e9e9b4"}
Sep 08 15:45:14.683138 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.683099    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:45:14.683722 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.683702    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:45:14.698166 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:14.698126    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-42x9f" podStartSLOduration=5.568614607 podStartE2EDuration="23.698109449s" totalImagesPullingTime="18.129494842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:13.211406753 +0000 UTC m=+22.707814720" watchObservedRunningTime="2026-09-08 15:45:14.698109449 +0000 UTC m=+24.194517417"
Sep 08 15:45:15.094766 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:15.094683    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:15.094907 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:15.094696    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:15.094907 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:15.094796    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:15.094907 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:15.094877    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:15.195209 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:15.195174    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"dbffa0900afff8bda18aec6f83e2ddd089ab186405cf21d6efb89c723fa3bb27"}
Sep 08 15:45:15.195666 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:15.195420    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:45:15.195978 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:15.195955    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xd4bw"
Sep 08 15:45:16.095231 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:16.095198    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:16.095456 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:16.095330    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:17.098846 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.098633    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:17.099436 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.098647    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:17.099436 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:17.098949    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:17.099436 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:17.099122    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:17.203374 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.203341    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" event={"ID":"e9788893-de8d-4fef-bcde-772bd40a940e","Type":"ContainerStarted","Data":"69044a0421a1629a3a67d0f647cbe1ae6a4f5352b10c3c24b45d818a2e61f3f3"}
Sep 08 15:45:17.203981 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.203849    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:45:17.203981 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.203920    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:45:17.221640 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.221613    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:45:17.222420 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.222396    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:45:17.230543 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:17.230495    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd" podStartSLOduration=8.067876187 podStartE2EDuration="26.230482051s" totalImagesPullingTime="18.162605864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:17.229579269 +0000 UTC m=+26.725987262" watchObservedRunningTime="2026-09-08 15:45:17.230482051 +0000 UTC m=+26.726890017"
Sep 08 15:45:18.095337 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:18.095299    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:18.095513 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:18.095419    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:18.207692 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:18.207651    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d" event={"ID":"6b67e8db-d511-4e40-8974-d97cfdce85ed","Type":"ContainerStarted","Data":"12c802058ee6f5c832268f4702a9f57a7a318a7c0708c62a54c8f26889c0cdbb"}
Sep 08 15:45:18.208377 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:18.207736    2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 15:45:18.227129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:18.227081    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thj8d" podStartSLOduration=3.654582715 podStartE2EDuration="27.227059901s" totalImagesPullingTime="23.572477186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:18.22667917 +0000 UTC m=+27.723087138" watchObservedRunningTime="2026-09-08 15:45:18.227059901 +0000 UTC m=+27.723467871"
Sep 08 15:45:18.869017 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:18.868975    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:45:19.095550 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:19.095496    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:19.095718 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:19.095631    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:19.095718 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:19.095681    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:19.095834 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:19.095788    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:20.105826 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:20.105790    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:20.106366 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:20.105894    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:20.213679 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:20.213592    2552 generic.go:356] "Generic (PLEG): container finished" podID="5d237297-4138-4c04-9495-cccbbff1e35e" containerID="5625be365f50ddcd37af7d87a9a557acb61e6da5ac270b271dc8f23dfac3a0a4" exitCode=0
Sep 08 15:45:20.213831 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:20.213682    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerDied","Data":"5625be365f50ddcd37af7d87a9a557acb61e6da5ac270b271dc8f23dfac3a0a4"}
Sep 08 15:45:21.095317 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:21.095289    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:21.095474 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:21.095357    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:21.095474 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:21.095374    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:21.095550 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:21.095470    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:22.095340 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:22.095266    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:22.095678 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:22.095370    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:22.220723 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:22.220693    2552 generic.go:356] "Generic (PLEG): container finished" podID="5d237297-4138-4c04-9495-cccbbff1e35e" containerID="f665c3df629f56159ef9cd765ed3d73d078d53cd6f5dbb57faf6d9a5a346a10c" exitCode=0
Sep 08 15:45:22.220877 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:22.220741    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerDied","Data":"f665c3df629f56159ef9cd765ed3d73d078d53cd6f5dbb57faf6d9a5a346a10c"}
Sep 08 15:45:23.095638 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:23.095601    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:23.096013 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:23.095601    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:23.096013 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:23.095722    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:23.096013 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:23.095762    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:23.225749 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:23.225664    2552 generic.go:356] "Generic (PLEG): container finished" podID="5d237297-4138-4c04-9495-cccbbff1e35e" containerID="63c89430196615eeb212af54d16a8b5b0988e760d514165477e4599f5613489f" exitCode=0
Sep 08 15:45:23.225749 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:23.225721    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerDied","Data":"63c89430196615eeb212af54d16a8b5b0988e760d514165477e4599f5613489f"}
Sep 08 15:45:24.095369 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:24.095336    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:24.095521 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.095437    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:24.602452 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:24.602413    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:24.602905 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.602585    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:45:24.602905 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:24.602598    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:24.602905 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.602649    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs podName:37acb33a-d9a0-484e-94ed-69d21bbb302e nodeName:}" failed. No retries permitted until 2026-09-08 15:45:56.602630769 +0000 UTC m=+66.099038719 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs") pod "network-metrics-daemon-9jph2" (UID: "37acb33a-d9a0-484e-94ed-69d21bbb302e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:45:24.602905 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.602732    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:45:24.602905 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.602797    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret podName:80e81107-5b70-4123-87bf-8c0827d277d2 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:56.60277842 +0000 UTC m=+66.099186390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret") pod "global-pull-secret-syncer-wqb2w" (UID: "80e81107-5b70-4123-87bf-8c0827d277d2") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:45:24.803789 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:24.803753    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:24.803952 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.803935    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:45:24.803996 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.803961    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:45:24.803996 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.803975    2552 projected.go:196] Error preparing data for projected volume kube-api-access-f72d9 for pod openshift-network-diagnostics/network-check-target-dmf8l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:45:24.804119 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:24.804071    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9 podName:0ab39284-a215-4381-b79a-faf6edfb5cb0 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:56.804056143 +0000 UTC m=+66.300464093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f72d9" (UniqueName: "kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9") pod "network-check-target-dmf8l" (UID: "0ab39284-a215-4381-b79a-faf6edfb5cb0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:45:25.095675 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.095088    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:25.095675 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:25.095453    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:25.095675 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.095088    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:25.095675 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:25.095632    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:25.382598 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.382561    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wqb2w"]
Sep 08 15:45:25.383499 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.382682    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:25.383499 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:25.382784    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:25.383499 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.383442    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jph2"]
Sep 08 15:45:25.383696 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.383537    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:25.383696 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:25.383643    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:25.395963 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.395796    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dmf8l"]
Sep 08 15:45:25.395963 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:25.395899    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:25.396342 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:25.395994    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:27.095346 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:27.095307    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:27.095773 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:27.095312    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:27.095773 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:27.095448    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:27.095773 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:27.095469    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:27.095773 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:27.095561    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:27.095773 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:27.095639    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:28.240720 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:28.240536    2552 generic.go:356] "Generic (PLEG): container finished" podID="5d237297-4138-4c04-9495-cccbbff1e35e" containerID="b23b468210c0d1036df37288230d1e3d0e0c2213db0a5b7d7e8329ee72347e57" exitCode=0
Sep 08 15:45:28.240720 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:28.240615    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerDied","Data":"b23b468210c0d1036df37288230d1e3d0e0c2213db0a5b7d7e8329ee72347e57"}
Sep 08 15:45:29.095487 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:29.095452    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:29.095487 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:29.095486    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:29.095682 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:29.095452    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:29.095682 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:29.095569    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dmf8l" podUID="0ab39284-a215-4381-b79a-faf6edfb5cb0"
Sep 08 15:45:29.095682 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:29.095625    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqb2w" podUID="80e81107-5b70-4123-87bf-8c0827d277d2"
Sep 08 15:45:29.095778 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:29.095712    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9jph2" podUID="37acb33a-d9a0-484e-94ed-69d21bbb302e"
Sep 08 15:45:29.245579 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:29.245551    2552 generic.go:356] "Generic (PLEG): container finished" podID="5d237297-4138-4c04-9495-cccbbff1e35e" containerID="1402b9bf5be4be286c68f19ba03b0c537dc73cad9bde1f79c0c964b809a631e9" exitCode=0
Sep 08 15:45:29.245894 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:29.245611    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerDied","Data":"1402b9bf5be4be286c68f19ba03b0c537dc73cad9bde1f79c0c964b809a631e9"}
Sep 08 15:45:30.251407 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.251377    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" event={"ID":"5d237297-4138-4c04-9495-cccbbff1e35e","Type":"ContainerStarted","Data":"f101f6b0683bb2a06c4b682bc0191f4e69a0641cce339c13b1859c87e1f57e7a"}
Sep 08 15:45:30.267953 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.267930    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-224.ec2.internal" event="NodeReady"
Sep 08 15:45:30.268093 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.268082    2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 15:45:30.275000 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.274964    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j5xsk" podStartSLOduration=7.159539432 podStartE2EDuration="39.274950171s" totalImagesPullingTime="32.115410739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:30.274195508 +0000 UTC m=+39.770603475" watchObservedRunningTime="2026-09-08 15:45:30.274950171 +0000 UTC m=+39.771358138"
Sep 08 15:45:30.317180 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.317113    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gxr6z"]
Sep 08 15:45:30.332310 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.332285    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jbvz5"]
Sep 08 15:45:30.332479 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.332422    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.334644 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.334615    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 15:45:30.334979 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.334961    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z2m58\""
Sep 08 15:45:30.335085 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.335067    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 15:45:30.335210 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.335192    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 15:45:30.352399 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.352377    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxr6z"]
Sep 08 15:45:30.352482 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.352406    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jbvz5"]
Sep 08 15:45:30.352482 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.352435    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k9xkq"]
Sep 08 15:45:30.352596 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.352579    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.354914 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.354889    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rhb4d\""
Sep 08 15:45:30.354914 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.354907    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 15:45:30.355082 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.354893    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 15:45:30.355082 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.354893    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 15:45:30.355082 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.354925    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 15:45:30.374150 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.374127    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k9xkq"]
Sep 08 15:45:30.374247 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.374215    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.377333 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.377316    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 15:45:30.377426 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.377317    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qhh8b\""
Sep 08 15:45:30.377426 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.377317    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 15:45:30.442280 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442242    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06ae3d7a-f203-4581-bb84-4f732af55a4a-cert\") pod \"ingress-canary-gxr6z\" (UID: \"06ae3d7a-f203-4581-bb84-4f732af55a4a\") " pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.442280 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442282    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jrm9\" (UniqueName: \"kubernetes.io/projected/06ae3d7a-f203-4581-bb84-4f732af55a4a-kube-api-access-9jrm9\") pod \"ingress-canary-gxr6z\" (UID: \"06ae3d7a-f203-4581-bb84-4f732af55a4a\") " pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.442464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442304    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e3ac920-58a2-4334-9794-0e0a05753954-crio-socket\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.442464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442378    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e3ac920-58a2-4334-9794-0e0a05753954-data-volume\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.442464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442421    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh5fr\" (UniqueName: \"kubernetes.io/projected/1e3ac920-58a2-4334-9794-0e0a05753954-kube-api-access-xh5fr\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.442556 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bddd1262-8293-4b27-ba17-153260e2cb4a-metrics-tls\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.442556 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442489    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e3ac920-58a2-4334-9794-0e0a05753954-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.442556 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442504    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e3ac920-58a2-4334-9794-0e0a05753954-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.442556 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442519    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bddd1262-8293-4b27-ba17-153260e2cb4a-config-volume\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.442666 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442586    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bddd1262-8293-4b27-ba17-153260e2cb4a-tmp-dir\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.442666 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.442612    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5d49\" (UniqueName: \"kubernetes.io/projected/bddd1262-8293-4b27-ba17-153260e2cb4a-kube-api-access-q5d49\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.533579 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.533544    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-frxpq"]
Sep 08 15:45:30.542875 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.542851    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bddd1262-8293-4b27-ba17-153260e2cb4a-metrics-tls\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.542961 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.542882    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e3ac920-58a2-4334-9794-0e0a05753954-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.542961 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.542898    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e3ac920-58a2-4334-9794-0e0a05753954-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.542961 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.542915    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bddd1262-8293-4b27-ba17-153260e2cb4a-config-volume\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.543129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.542962    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bddd1262-8293-4b27-ba17-153260e2cb4a-tmp-dir\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.543129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.542985    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5d49\" (UniqueName: \"kubernetes.io/projected/bddd1262-8293-4b27-ba17-153260e2cb4a-kube-api-access-q5d49\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.543129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543012    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06ae3d7a-f203-4581-bb84-4f732af55a4a-cert\") pod \"ingress-canary-gxr6z\" (UID: \"06ae3d7a-f203-4581-bb84-4f732af55a4a\") " pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.543129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543057    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jrm9\" (UniqueName: \"kubernetes.io/projected/06ae3d7a-f203-4581-bb84-4f732af55a4a-kube-api-access-9jrm9\") pod \"ingress-canary-gxr6z\" (UID: \"06ae3d7a-f203-4581-bb84-4f732af55a4a\") " pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.543129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543084    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e3ac920-58a2-4334-9794-0e0a05753954-crio-socket\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.543129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543108    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e3ac920-58a2-4334-9794-0e0a05753954-data-volume\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.543401 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543131    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh5fr\" (UniqueName: \"kubernetes.io/projected/1e3ac920-58a2-4334-9794-0e0a05753954-kube-api-access-xh5fr\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.543401 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543319    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bddd1262-8293-4b27-ba17-153260e2cb4a-tmp-dir\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.543561 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543538    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e3ac920-58a2-4334-9794-0e0a05753954-crio-socket\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.543618 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543538    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e3ac920-58a2-4334-9794-0e0a05753954-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.543700 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543677    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bddd1262-8293-4b27-ba17-153260e2cb4a-config-volume\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.543770 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.543751    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e3ac920-58a2-4334-9794-0e0a05753954-data-volume\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.547070 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.547050    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e3ac920-58a2-4334-9794-0e0a05753954-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.547182 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.547096    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bddd1262-8293-4b27-ba17-153260e2cb4a-metrics-tls\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.547237 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.547212    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06ae3d7a-f203-4581-bb84-4f732af55a4a-cert\") pod \"ingress-canary-gxr6z\" (UID: \"06ae3d7a-f203-4581-bb84-4f732af55a4a\") " pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.549596 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.549576    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-frxpq"]
Sep 08 15:45:30.549680 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.549664    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.552348 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.552164    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 08 15:45:30.552448 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.552332    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 08 15:45:30.553039 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.552870    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-77m5k\""
Sep 08 15:45:30.553039 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.552932    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh5fr\" (UniqueName: \"kubernetes.io/projected/1e3ac920-58a2-4334-9794-0e0a05753954-kube-api-access-xh5fr\") pod \"insights-runtime-extractor-jbvz5\" (UID: \"1e3ac920-58a2-4334-9794-0e0a05753954\") " pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.553039 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.552883    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 15:45:30.553039 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.552998    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 15:45:30.553282 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.552882    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 15:45:30.553370 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.553343    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jrm9\" (UniqueName: \"kubernetes.io/projected/06ae3d7a-f203-4581-bb84-4f732af55a4a-kube-api-access-9jrm9\") pod \"ingress-canary-gxr6z\" (UID: \"06ae3d7a-f203-4581-bb84-4f732af55a4a\") " pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.553996 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.553973    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5d49\" (UniqueName: \"kubernetes.io/projected/bddd1262-8293-4b27-ba17-153260e2cb4a-kube-api-access-q5d49\") pod \"dns-default-k9xkq\" (UID: \"bddd1262-8293-4b27-ba17-153260e2cb4a\") " pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.642446 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.642408    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxr6z"
Sep 08 15:45:30.643652 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.643623    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4jxk\" (UniqueName: \"kubernetes.io/projected/afc3611e-8b39-4918-99ed-8121d46112d6-kube-api-access-l4jxk\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.643757 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.643679    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.643757 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.643731    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afc3611e-8b39-4918-99ed-8121d46112d6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.643847 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.643777    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.661663 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.661641    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jbvz5"
Sep 08 15:45:30.681860 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.681839    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:30.744252 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.744176    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afc3611e-8b39-4918-99ed-8121d46112d6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.744252 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.744239    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.744378 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.744320    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jxk\" (UniqueName: \"kubernetes.io/projected/afc3611e-8b39-4918-99ed-8121d46112d6-kube-api-access-l4jxk\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.744429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.744380    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.745964 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:30.745770    2552 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 08 15:45:30.745964 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:30.745860    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-tls podName:afc3611e-8b39-4918-99ed-8121d46112d6 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:31.245838112 +0000 UTC m=+40.742246080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-frxpq" (UID: "afc3611e-8b39-4918-99ed-8121d46112d6") : secret "prometheus-operator-tls" not found
Sep 08 15:45:30.747596 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.747496    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afc3611e-8b39-4918-99ed-8121d46112d6-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.749781 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.749752    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.767761 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.766841    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4jxk\" (UniqueName: \"kubernetes.io/projected/afc3611e-8b39-4918-99ed-8121d46112d6-kube-api-access-l4jxk\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:30.967470 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.967389    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxr6z"]
Sep 08 15:45:30.970360 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:30.970331    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jbvz5"]
Sep 08 15:45:30.970674 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:30.970654    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ae3d7a_f203_4581_bb84_4f732af55a4a.slice/crio-3bff3c0510c6b8def086e3619a52c12f8f2cb879c3abe55e6dc6bdcb134dcc41 WatchSource:0}: Error finding container 3bff3c0510c6b8def086e3619a52c12f8f2cb879c3abe55e6dc6bdcb134dcc41: Status 404 returned error can't find the container with id 3bff3c0510c6b8def086e3619a52c12f8f2cb879c3abe55e6dc6bdcb134dcc41
Sep 08 15:45:30.972967 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:30.972944    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ac920_58a2_4334_9794_0e0a05753954.slice/crio-07f0b46bc8148aa1d576e3f36c988029e2d4f06926263fbee62ed81445229001 WatchSource:0}: Error finding container 07f0b46bc8148aa1d576e3f36c988029e2d4f06926263fbee62ed81445229001: Status 404 returned error can't find the container with id 07f0b46bc8148aa1d576e3f36c988029e2d4f06926263fbee62ed81445229001
Sep 08 15:45:31.002785 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.002751    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k9xkq"]
Sep 08 15:45:31.005238 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:31.005221    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbddd1262_8293_4b27_ba17_153260e2cb4a.slice/crio-a121029041cb12891a9c0fda55b90d8b4993c09fed353730cbb96b887d2cac0d WatchSource:0}: Error finding container a121029041cb12891a9c0fda55b90d8b4993c09fed353730cbb96b887d2cac0d: Status 404 returned error can't find the container with id a121029041cb12891a9c0fda55b90d8b4993c09fed353730cbb96b887d2cac0d
Sep 08 15:45:31.095375 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.095350    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:31.095601 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.095552    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:31.095724 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.095705    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:31.098056 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.098013    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 15:45:31.098173 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.098074    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-78hmt\""
Sep 08 15:45:31.098173 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.098127    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 15:45:31.098283 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.098185    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xkdlq\""
Sep 08 15:45:31.098429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.098413    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 15:45:31.098477 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.098449    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 15:45:31.248451 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.248366    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:31.250646 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.250621    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/afc3611e-8b39-4918-99ed-8121d46112d6-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-frxpq\" (UID: \"afc3611e-8b39-4918-99ed-8121d46112d6\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:31.254249 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.254223    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k9xkq" event={"ID":"bddd1262-8293-4b27-ba17-153260e2cb4a","Type":"ContainerStarted","Data":"a121029041cb12891a9c0fda55b90d8b4993c09fed353730cbb96b887d2cac0d"}
Sep 08 15:45:31.255670 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.255645    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbvz5" event={"ID":"1e3ac920-58a2-4334-9794-0e0a05753954","Type":"ContainerStarted","Data":"89968b95e3a2df3ed3896cf72bdc7b81a177dd187576afe632d859b0dbd3eb12"}
Sep 08 15:45:31.255769 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.255699    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbvz5" event={"ID":"1e3ac920-58a2-4334-9794-0e0a05753954","Type":"ContainerStarted","Data":"07f0b46bc8148aa1d576e3f36c988029e2d4f06926263fbee62ed81445229001"}
Sep 08 15:45:31.256745 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.256723    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxr6z" event={"ID":"06ae3d7a-f203-4581-bb84-4f732af55a4a","Type":"ContainerStarted","Data":"3bff3c0510c6b8def086e3619a52c12f8f2cb879c3abe55e6dc6bdcb134dcc41"}
Sep 08 15:45:31.467639 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.467589    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq"
Sep 08 15:45:31.768075 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:31.767976    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-frxpq"]
Sep 08 15:45:31.771330 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:31.771298    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc3611e_8b39_4918_99ed_8121d46112d6.slice/crio-da9405adb568d333aa5fdd052e52b48fab4841cc2a5a66407250a0099a7085df WatchSource:0}: Error finding container da9405adb568d333aa5fdd052e52b48fab4841cc2a5a66407250a0099a7085df: Status 404 returned error can't find the container with id da9405adb568d333aa5fdd052e52b48fab4841cc2a5a66407250a0099a7085df
Sep 08 15:45:32.261727 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:32.261680    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq" event={"ID":"afc3611e-8b39-4918-99ed-8121d46112d6","Type":"ContainerStarted","Data":"da9405adb568d333aa5fdd052e52b48fab4841cc2a5a66407250a0099a7085df"}
Sep 08 15:45:34.665345 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.665310    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ffc47769-shw9z"]
Sep 08 15:45:34.700423 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.700388    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ffc47769-shw9z"]
Sep 08 15:45:34.700590 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.700546    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.703144 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.703117    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 15:45:34.703472 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.703442    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 15:45:34.703592 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.703493    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 08 15:45:34.703592 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.703539    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 15:45:34.704184 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.703973    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-p89kc\""
Sep 08 15:45:34.704184 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.704004    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 15:45:34.704184 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.704075    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 08 15:45:34.704398 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.704268    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 15:45:34.709082 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.708853    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 15:45:34.779568 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.779534    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-serving-cert\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.779568 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.779571    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-config\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.779750 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.779590    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-service-ca\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.779750 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.779703    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkfvd\" (UniqueName: \"kubernetes.io/projected/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-kube-api-access-lkfvd\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.779818 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.779762    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-oauth-config\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.779818 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.779786    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-trusted-ca-bundle\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.779818 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.779809    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-oauth-serving-cert\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.880609 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.880576    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkfvd\" (UniqueName: \"kubernetes.io/projected/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-kube-api-access-lkfvd\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.880770 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.880626    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-oauth-config\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.880770 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.880650    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-trusted-ca-bundle\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.880770 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.880672    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-oauth-serving-cert\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.880770 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.880723    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-serving-cert\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.880770 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.880745    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-config\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.881008 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.880864    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-service-ca\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.881523 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.881499    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-oauth-serving-cert\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.881620 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.881540    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-config\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.881657 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.881622    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-service-ca\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.881693 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.881628    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-trusted-ca-bundle\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.883038 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.883012    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-oauth-config\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.883220 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.883202    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-serving-cert\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:34.892413 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:34.892383    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkfvd\" (UniqueName: \"kubernetes.io/projected/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-kube-api-access-lkfvd\") pod \"console-7ffc47769-shw9z\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") " pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:35.012915 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:35.012887    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:35.495298 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:35.495165    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ffc47769-shw9z"]
Sep 08 15:45:35.500548 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:35.500513    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b6d08cd_35c6_461d_9ca1_87d22ab72fb6.slice/crio-e94b1cdcab416c59126179b55fe2ea942a609f6c96aece6efb97a24069ca6d84 WatchSource:0}: Error finding container e94b1cdcab416c59126179b55fe2ea942a609f6c96aece6efb97a24069ca6d84: Status 404 returned error can't find the container with id e94b1cdcab416c59126179b55fe2ea942a609f6c96aece6efb97a24069ca6d84
Sep 08 15:45:36.279913 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.279866    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k9xkq" event={"ID":"bddd1262-8293-4b27-ba17-153260e2cb4a","Type":"ContainerStarted","Data":"ab1aa2638d635dceb582253699c9648f60b1a1a81f828aec56b87f2f3e506f10"}
Sep 08 15:45:36.280421 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.279929    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k9xkq" event={"ID":"bddd1262-8293-4b27-ba17-153260e2cb4a","Type":"ContainerStarted","Data":"5b648506fe03a835ddb5a683f243c40b0f6e050bf6a7c1eeda0672dcd5fa1729"}
Sep 08 15:45:36.280421 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.280226    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:36.281848 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.281816    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffc47769-shw9z" event={"ID":"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6","Type":"ContainerStarted","Data":"e94b1cdcab416c59126179b55fe2ea942a609f6c96aece6efb97a24069ca6d84"}
Sep 08 15:45:36.284067 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.284013    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbvz5" event={"ID":"1e3ac920-58a2-4334-9794-0e0a05753954","Type":"ContainerStarted","Data":"efac9e36387ea9c7c86d620f661152257d696042039afa5447991a24099572da"}
Sep 08 15:45:36.286295 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.286269    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq" event={"ID":"afc3611e-8b39-4918-99ed-8121d46112d6","Type":"ContainerStarted","Data":"2da1495dbd7ae317d7018a71bf5c2527a97c98087d9380c75e04f40237ffb371"}
Sep 08 15:45:36.286409 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.286324    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq" event={"ID":"afc3611e-8b39-4918-99ed-8121d46112d6","Type":"ContainerStarted","Data":"88fac192a549c001c6a6c143e19d320f5e4019c03fbbb7e953b8c8e72a1f3679"}
Sep 08 15:45:36.288379 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.288338    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxr6z" event={"ID":"06ae3d7a-f203-4581-bb84-4f732af55a4a","Type":"ContainerStarted","Data":"1da846c1d2c802a7dc92cc12732e1432a34ecb01166d2f9b15bf8da4dece59ca"}
Sep 08 15:45:36.299145 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.299069    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k9xkq" podStartSLOduration=2.078432233 podStartE2EDuration="6.299052103s" totalImagesPullingTime="4.22061987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:36.297705424 +0000 UTC m=+45.794113392" watchObservedRunningTime="2026-09-08 15:45:36.299052103 +0000 UTC m=+45.795460063"
Sep 08 15:45:36.313611 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.313554    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gxr6z" podStartSLOduration=2.06037913 podStartE2EDuration="6.31353752s" totalImagesPullingTime="4.25315839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:36.312913557 +0000 UTC m=+45.809321524" watchObservedRunningTime="2026-09-08 15:45:36.31353752 +0000 UTC m=+45.809945488"
Sep 08 15:45:36.329705 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:36.329635    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-frxpq" podStartSLOduration=2.877795333 podStartE2EDuration="6.329621805s" totalImagesPullingTime="3.451826472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:36.329410297 +0000 UTC m=+45.825818338" watchObservedRunningTime="2026-09-08 15:45:36.329621805 +0000 UTC m=+45.826029771"
Sep 08 15:45:37.906238 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.906203    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"]
Sep 08 15:45:37.933938 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.933389    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"]
Sep 08 15:45:37.933938 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.933443    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"]
Sep 08 15:45:37.933938 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.933521    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:37.935949 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.935920    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 08 15:45:37.936096 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.936010    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 08 15:45:37.936149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.936140    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-56qd2\""
Sep 08 15:45:37.961942 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.961310    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6bkmd"]
Sep 08 15:45:37.962236 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.962212    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:37.965919 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.965896    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 08 15:45:37.966292 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.966273    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-msgks\""
Sep 08 15:45:37.966555 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.966536    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 08 15:45:37.966937 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.966915    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 08 15:45:37.993327 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.993301    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"]
Sep 08 15:45:37.993466 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.993442    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:37.996599 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.996577    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 15:45:37.996599 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.996579    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 15:45:37.996785 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.996582    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nkrhj\""
Sep 08 15:45:37.996785 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:37.996628    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 15:45:38.107986 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.107948    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.108183 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.107995    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.108183 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108043    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.108183 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108069    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108183 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108097    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22frd\" (UniqueName: \"kubernetes.io/projected/4ae14a40-5818-48f5-a879-1e7b8520ced1-kube-api-access-22frd\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108183 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108117    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-wtmp\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108196    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.108454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108235    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe255b2-94df-4704-b7d2-91ca9c419a28-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.108454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108285    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-sys\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108301    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-root\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108316    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-tls\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108347    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.108454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108439    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ae14a40-5818-48f5-a879-1e7b8520ced1-metrics-client-ca\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108471    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5lp5\" (UniqueName: \"kubernetes.io/projected/bbe255b2-94df-4704-b7d2-91ca9c419a28-kube-api-access-q5lp5\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.108814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108492    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.108814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108530    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108563    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.108814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108584    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-textfile\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.108814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.108608    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c82zd\" (UniqueName: \"kubernetes.io/projected/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-api-access-c82zd\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.209870 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.209795    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.209870 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.209835    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.209870 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.209853    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.209879    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.209938    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22frd\" (UniqueName: \"kubernetes.io/projected/4ae14a40-5818-48f5-a879-1e7b8520ced1-kube-api-access-22frd\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:38.209965    2552 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.209989    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-wtmp\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:38.210063    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-tls podName:56fb494c-88ee-4cd2-bb69-93e1d5e5e507 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:38.710043004 +0000 UTC m=+48.206450962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-8l6qg" (UID: "56fb494c-88ee-4cd2-bb69-93e1d5e5e507") : secret "kube-state-metrics-tls" not found
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210084    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210116    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe255b2-94df-4704-b7d2-91ca9c419a28-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.210149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210149    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-sys\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210173    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-root\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210200    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-tls\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:38.210255    2552 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210257    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210290    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ae14a40-5818-48f5-a879-1e7b8520ced1-metrics-client-ca\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:38.210310    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-tls podName:bbe255b2-94df-4704-b7d2-91ca9c419a28 nodeName:}" failed. No retries permitted until 2026-09-08 15:45:38.710293189 +0000 UTC m=+48.206701149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-nt7hb" (UID: "bbe255b2-94df-4704-b7d2-91ca9c419a28") : secret "openshift-state-metrics-tls" not found
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210342    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-root\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210172    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-wtmp\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210407    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5lp5\" (UniqueName: \"kubernetes.io/projected/bbe255b2-94df-4704-b7d2-91ca9c419a28-kube-api-access-q5lp5\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210453    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210497    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.210532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210532    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.211110 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210566    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-textfile\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.211110 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210595    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c82zd\" (UniqueName: \"kubernetes.io/projected/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-api-access-c82zd\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.211110 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210655    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.211110 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.210984    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ae14a40-5818-48f5-a879-1e7b8520ced1-metrics-client-ca\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.211329 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.211135    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.211329 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.211135    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbe255b2-94df-4704-b7d2-91ca9c419a28-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.211329 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.211177    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ae14a40-5818-48f5-a879-1e7b8520ced1-sys\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.211329 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.211276    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.211329 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.211300    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.211589 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.211434    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-textfile\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.213503 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.213472    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.213622 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.213600    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.213682 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.213609    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ae14a40-5818-48f5-a879-1e7b8520ced1-node-exporter-tls\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.213873 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.213850    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.218493 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.218469    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22frd\" (UniqueName: \"kubernetes.io/projected/4ae14a40-5818-48f5-a879-1e7b8520ced1-kube-api-access-22frd\") pod \"node-exporter-6bkmd\" (UID: \"4ae14a40-5818-48f5-a879-1e7b8520ced1\") " pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.219011 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.218982    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c82zd\" (UniqueName: \"kubernetes.io/projected/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-api-access-c82zd\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.219526 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.219507    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5lp5\" (UniqueName: \"kubernetes.io/projected/bbe255b2-94df-4704-b7d2-91ca9c419a28-kube-api-access-q5lp5\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.303981 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.303946    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6bkmd"
Sep 08 15:45:38.497812 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:38.497783    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae14a40_5818_48f5_a879_1e7b8520ced1.slice/crio-8a1c6eacd7b96579d42b1eac3a9cf1c6cb9326001392d96409cd341324256f39 WatchSource:0}: Error finding container 8a1c6eacd7b96579d42b1eac3a9cf1c6cb9326001392d96409cd341324256f39: Status 404 returned error can't find the container with id 8a1c6eacd7b96579d42b1eac3a9cf1c6cb9326001392d96409cd341324256f39
Sep 08 15:45:38.714411 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.714376    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.714554 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.714495    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.716807 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.716776    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbe255b2-94df-4704-b7d2-91ca9c419a28-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-nt7hb\" (UID: \"bbe255b2-94df-4704-b7d2-91ca9c419a28\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.716888 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.716870    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/56fb494c-88ee-4cd2-bb69-93e1d5e5e507-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8l6qg\" (UID: \"56fb494c-88ee-4cd2-bb69-93e1d5e5e507\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:38.844959 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.844882    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"
Sep 08 15:45:38.872879 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:38.872847    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"
Sep 08 15:45:39.152681 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.152637    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb"]
Sep 08 15:45:39.158256 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:39.158226    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe255b2_94df_4704_b7d2_91ca9c419a28.slice/crio-22759dba5ddf0df1dba63c1cfef24962f3fdb85557f78cb329563a593ded68a2 WatchSource:0}: Error finding container 22759dba5ddf0df1dba63c1cfef24962f3fdb85557f78cb329563a593ded68a2: Status 404 returned error can't find the container with id 22759dba5ddf0df1dba63c1cfef24962f3fdb85557f78cb329563a593ded68a2
Sep 08 15:45:39.161106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.161077    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg"]
Sep 08 15:45:39.164050 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:39.164004    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fb494c_88ee_4cd2_bb69_93e1d5e5e507.slice/crio-9465efcc901c3bd4c1fb5a94cb1d777834a51224e97f3799837b945fab359a5f WatchSource:0}: Error finding container 9465efcc901c3bd4c1fb5a94cb1d777834a51224e97f3799837b945fab359a5f: Status 404 returned error can't find the container with id 9465efcc901c3bd4c1fb5a94cb1d777834a51224e97f3799837b945fab359a5f
Sep 08 15:45:39.304448 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.304413    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbvz5" event={"ID":"1e3ac920-58a2-4334-9794-0e0a05753954","Type":"ContainerStarted","Data":"2af3cfc3b3178aa1ed827363d78d5f6bd060fcef79d2aea1615010fd59a695be"}
Sep 08 15:45:39.306047 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.305994    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6bkmd" event={"ID":"4ae14a40-5818-48f5-a879-1e7b8520ced1","Type":"ContainerStarted","Data":"8a1c6eacd7b96579d42b1eac3a9cf1c6cb9326001392d96409cd341324256f39"}
Sep 08 15:45:39.307465 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.307435    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg" event={"ID":"56fb494c-88ee-4cd2-bb69-93e1d5e5e507","Type":"ContainerStarted","Data":"9465efcc901c3bd4c1fb5a94cb1d777834a51224e97f3799837b945fab359a5f"}
Sep 08 15:45:39.310398 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.310162    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb" event={"ID":"bbe255b2-94df-4704-b7d2-91ca9c419a28","Type":"ContainerStarted","Data":"30930ff1b2818bf54e3f634ead95f382404a0eccfaace4bf95e305bf60edce1e"}
Sep 08 15:45:39.310398 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.310378    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb" event={"ID":"bbe255b2-94df-4704-b7d2-91ca9c419a28","Type":"ContainerStarted","Data":"22759dba5ddf0df1dba63c1cfef24962f3fdb85557f78cb329563a593ded68a2"}
Sep 08 15:45:39.312167 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.312144    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffc47769-shw9z" event={"ID":"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6","Type":"ContainerStarted","Data":"4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc"}
Sep 08 15:45:39.347741 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.347701    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jbvz5" podStartSLOduration=2.146853856 podStartE2EDuration="9.347686442s" totalImagesPullingTime="7.200832586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:45:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:39.321242141 +0000 UTC m=+48.817650158" watchObservedRunningTime="2026-09-08 15:45:39.347686442 +0000 UTC m=+48.844094406"
Sep 08 15:45:39.348153 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:39.348128    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ffc47769-shw9z" podStartSLOduration=1.9622444300000002 podStartE2EDuration="5.348122721s" totalImagesPullingTime="3.385878291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:39.347813247 +0000 UTC m=+48.844221196" watchObservedRunningTime="2026-09-08 15:45:39.348122721 +0000 UTC m=+48.844530741"
Sep 08 15:45:40.318354 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.318233    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb" event={"ID":"bbe255b2-94df-4704-b7d2-91ca9c419a28","Type":"ContainerStarted","Data":"d40caada9a8d73032105cdcf1cc0e94f9f7a7a7d273a77ab4d8990927e402e81"}
Sep 08 15:45:40.320089 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.320058    2552 generic.go:356] "Generic (PLEG): container finished" podID="4ae14a40-5818-48f5-a879-1e7b8520ced1" containerID="5a6445d2818106e166ede00443700a2e7f52741e617ceb231d196c93f7d1799f" exitCode=0
Sep 08 15:45:40.320261 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.320157    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6bkmd" event={"ID":"4ae14a40-5818-48f5-a879-1e7b8520ced1","Type":"ContainerDied","Data":"5a6445d2818106e166ede00443700a2e7f52741e617ceb231d196c93f7d1799f"}
Sep 08 15:45:40.929760 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.929654    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b4df7cffb-ql774"]
Sep 08 15:45:40.939124 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.939097    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:40.943157 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.943129    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 08 15:45:40.943551 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.943437    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 08 15:45:40.943551 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.943504    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g8f8p\""
Sep 08 15:45:40.943551 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.943504    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 08 15:45:40.943830 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.943558    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 08 15:45:40.943830 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.943606    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 08 15:45:40.943830 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.943506    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9gvpnrcpova13\""
Sep 08 15:45:40.955884 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:40.955370    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b4df7cffb-ql774"]
Sep 08 15:45:41.034790 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.034762    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62e0dc58-e0f1-4071-baca-5fa054f1bd78-metrics-client-ca\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.034908 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.034826    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-tls\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.034966 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.034915    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.035022 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.034974    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.035022 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.034999    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.035119 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.035044    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.035119 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.035078    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn7tm\" (UniqueName: \"kubernetes.io/projected/62e0dc58-e0f1-4071-baca-5fa054f1bd78-kube-api-access-tn7tm\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.035119 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.035098    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-grpc-tls\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136304 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136272    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62e0dc58-e0f1-4071-baca-5fa054f1bd78-metrics-client-ca\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136444 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136403    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-tls\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136484 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136465    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136527 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136514    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136570 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136540    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136620 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136569    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136620 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136604    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn7tm\" (UniqueName: \"kubernetes.io/projected/62e0dc58-e0f1-4071-baca-5fa054f1bd78-kube-api-access-tn7tm\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136713 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136639    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-grpc-tls\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.136992 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.136966    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62e0dc58-e0f1-4071-baca-5fa054f1bd78-metrics-client-ca\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.140982 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.139128    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.140982 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.139542    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.140982 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.140053    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-tls\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.140982 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.140087    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.140982 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.140369    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.141477 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.141457    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/62e0dc58-e0f1-4071-baca-5fa054f1bd78-secret-grpc-tls\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.147501 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.147479    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn7tm\" (UniqueName: \"kubernetes.io/projected/62e0dc58-e0f1-4071-baca-5fa054f1bd78-kube-api-access-tn7tm\") pod \"thanos-querier-b4df7cffb-ql774\" (UID: \"62e0dc58-e0f1-4071-baca-5fa054f1bd78\") " pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.249751 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.249718    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:41.330799 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.330697    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6bkmd" event={"ID":"4ae14a40-5818-48f5-a879-1e7b8520ced1","Type":"ContainerStarted","Data":"1301cf258664d03c04552ab0bd82529f81ade25058ad69a9c20ea13b228186e6"}
Sep 08 15:45:41.330799 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.330756    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6bkmd" event={"ID":"4ae14a40-5818-48f5-a879-1e7b8520ced1","Type":"ContainerStarted","Data":"74bc51ea88c5f8781b6a6871eff01e3a121a14de2480dd4cabbc088aad1707cb"}
Sep 08 15:45:41.332749 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.332726    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg" event={"ID":"56fb494c-88ee-4cd2-bb69-93e1d5e5e507","Type":"ContainerStarted","Data":"0b76782d9e9f9cdeb4d50744c83904170a09294f5fe3ecf7d63a5ae6cc7101c0"}
Sep 08 15:45:41.335232 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.335207    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb" event={"ID":"bbe255b2-94df-4704-b7d2-91ca9c419a28","Type":"ContainerStarted","Data":"80cdf56e192897eca5ee3235077e3e9f792ff87713fe20f416ad6c29817831d2"}
Sep 08 15:45:41.351342 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.351280    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6bkmd" podStartSLOduration=2.696953512 podStartE2EDuration="4.351262355s" totalImagesPullingTime="1.563144294s" totalInitContainerRuntime="91.164549ms" isStatefulPod=true podCreationTimestamp="2026-09-08 15:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:41.350273796 +0000 UTC m=+50.846681764" watchObservedRunningTime="2026-09-08 15:45:41.351262355 +0000 UTC m=+50.847670323"
Sep 08 15:45:41.370301 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.370259    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-nt7hb" podStartSLOduration=2.561672605 podStartE2EDuration="4.370244874s" totalImagesPullingTime="1.808572269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:41.369164865 +0000 UTC m=+50.865572832" watchObservedRunningTime="2026-09-08 15:45:41.370244874 +0000 UTC m=+50.866652840"
Sep 08 15:45:41.546312 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:41.546287    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b4df7cffb-ql774"]
Sep 08 15:45:41.548337 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:41.548309    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62e0dc58_e0f1_4071_baca_5fa054f1bd78.slice/crio-daf7a51ba22e71d4815b98ff67b4377dfa6f1be43b5f72c96f9b828de71a79c6 WatchSource:0}: Error finding container daf7a51ba22e71d4815b98ff67b4377dfa6f1be43b5f72c96f9b828de71a79c6: Status 404 returned error can't find the container with id daf7a51ba22e71d4815b98ff67b4377dfa6f1be43b5f72c96f9b828de71a79c6
Sep 08 15:45:42.312473 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.312438    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5487ccc788-sw5tv"]
Sep 08 15:45:42.334680 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.334652    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5487ccc788-sw5tv"]
Sep 08 15:45:42.335070 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.334743    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.336934 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.336912    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 08 15:45:42.337715 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.337652    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 08 15:45:42.337871 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.337737    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-m5lqj\""
Sep 08 15:45:42.337871 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.337831    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 15:45:42.338084 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.337881    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 08 15:45:42.338084 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.337923    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ainmehp365koh\""
Sep 08 15:45:42.340975 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.340942    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg" event={"ID":"56fb494c-88ee-4cd2-bb69-93e1d5e5e507","Type":"ContainerStarted","Data":"53b35f633a5caf070b824c064e823e0e0563b233278ffb5315efc21f8ae92d72"}
Sep 08 15:45:42.341098 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.341017    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg" event={"ID":"56fb494c-88ee-4cd2-bb69-93e1d5e5e507","Type":"ContainerStarted","Data":"91055bc024f5ac77c4726cc78327c1c30ea9f43d15942d8e5d1da344c31692f3"}
Sep 08 15:45:42.342587 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.342296    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" event={"ID":"62e0dc58-e0f1-4071-baca-5fa054f1bd78","Type":"ContainerStarted","Data":"daf7a51ba22e71d4815b98ff67b4377dfa6f1be43b5f72c96f9b828de71a79c6"}
Sep 08 15:45:42.370074 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.370011    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8l6qg" podStartSLOduration=3.322033058 podStartE2EDuration="5.369990892s" totalImagesPullingTime="2.047957834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:42.368503793 +0000 UTC m=+51.864911761" watchObservedRunningTime="2026-09-08 15:45:42.369990892 +0000 UTC m=+51.866398861"
Sep 08 15:45:42.448843 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.448800    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ff809900-2d1e-405a-be92-f1233a4daa58-metrics-server-audit-profiles\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.448998 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.448879    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-client-ca-bundle\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.448998 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.448948    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-secret-metrics-server-tls\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.449409 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.449154    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ff809900-2d1e-405a-be92-f1233a4daa58-audit-log\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.449409 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.449192    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t457\" (UniqueName: \"kubernetes.io/projected/ff809900-2d1e-405a-be92-f1233a4daa58-kube-api-access-9t457\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.449409 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.449245    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-secret-metrics-server-client-certs\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.449816 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.449788    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff809900-2d1e-405a-be92-f1233a4daa58-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.550526 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550486    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-secret-metrics-server-tls\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.550707 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550558    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ff809900-2d1e-405a-be92-f1233a4daa58-audit-log\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.550707 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550585    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t457\" (UniqueName: \"kubernetes.io/projected/ff809900-2d1e-405a-be92-f1233a4daa58-kube-api-access-9t457\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.550707 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550614    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-secret-metrics-server-client-certs\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.550707 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550702    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff809900-2d1e-405a-be92-f1233a4daa58-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.550903 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550734    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ff809900-2d1e-405a-be92-f1233a4daa58-metrics-server-audit-profiles\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.550903 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550757    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-client-ca-bundle\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.551019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.550995    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ff809900-2d1e-405a-be92-f1233a4daa58-audit-log\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.551642 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.551605    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff809900-2d1e-405a-be92-f1233a4daa58-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.551938 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.551901    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ff809900-2d1e-405a-be92-f1233a4daa58-metrics-server-audit-profiles\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.553571 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.553526    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-client-ca-bundle\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.553757 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.553731    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-secret-metrics-server-client-certs\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.553900 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.553877    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ff809900-2d1e-405a-be92-f1233a4daa58-secret-metrics-server-tls\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.559253 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.559225    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t457\" (UniqueName: \"kubernetes.io/projected/ff809900-2d1e-405a-be92-f1233a4daa58-kube-api-access-9t457\") pod \"metrics-server-5487ccc788-sw5tv\" (UID: \"ff809900-2d1e-405a-be92-f1233a4daa58\") " pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.645552 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.645494    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:45:42.951664 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:42.951586    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5487ccc788-sw5tv"]
Sep 08 15:45:42.954666 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:42.954638    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff809900_2d1e_405a_be92_f1233a4daa58.slice/crio-f60d356e12fc644ee74e216f3c4d95e9678dbab35b914f9818042019dd77a6bd WatchSource:0}: Error finding container f60d356e12fc644ee74e216f3c4d95e9678dbab35b914f9818042019dd77a6bd: Status 404 returned error can't find the container with id f60d356e12fc644ee74e216f3c4d95e9678dbab35b914f9818042019dd77a6bd
Sep 08 15:45:43.348685 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:43.348588    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv" event={"ID":"ff809900-2d1e-405a-be92-f1233a4daa58","Type":"ContainerStarted","Data":"f60d356e12fc644ee74e216f3c4d95e9678dbab35b914f9818042019dd77a6bd"}
Sep 08 15:45:44.256720 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.256690    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:45:44.260281 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.260257    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.263872 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.263845    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 15:45:44.263995 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.263968    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 15:45:44.266429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.266238    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 15:45:44.266429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.266238    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 15:45:44.266429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.266296    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1d6hcf2fatp79\""
Sep 08 15:45:44.266661 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.266574    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 15:45:44.266773 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.266755    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d65qd\""
Sep 08 15:45:44.266942 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.266927    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 15:45:44.267011 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.266990    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 15:45:44.267134 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267117    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 15:45:44.267313 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267254    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 15:45:44.267402 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267314    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.267402 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267347    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.267667 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267374    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.267857 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267816    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 15:45:44.267963 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267863    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 15:45:44.268038 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267980    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 15:45:44.268101 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.267800    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-config\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.268156 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.268091    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.268763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.268214    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 15:45:44.268763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.268247    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.268763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.268287    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.268763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.268402    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.268763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.268437    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.269605 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.268495    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.269697 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.269666    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nrh6\" (UniqueName: \"kubernetes.io/projected/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-kube-api-access-7nrh6\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.270858 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.269723    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.270996 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.270960    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.271225 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.271077    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.271314 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.271257    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.271370 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.271339    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.271423 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.271377    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-web-config\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.271530 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.271515    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-config-out\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.271700 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.271550    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.271700 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.271646    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.276560 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.276257    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 15:45:44.286537 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.286416    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:45:44.355858 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.355818    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" event={"ID":"62e0dc58-e0f1-4071-baca-5fa054f1bd78","Type":"ContainerStarted","Data":"c7fabb5dee0682bba512135f93fb61588fd5260fd4798216cbb7f3191c34dc12"}
Sep 08 15:45:44.356365 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.355897    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" event={"ID":"62e0dc58-e0f1-4071-baca-5fa054f1bd78","Type":"ContainerStarted","Data":"3caf76a5eb6fa94cbad9c3e875c39fbb3b8445abe4458c47ea3ca456000d4815"}
Sep 08 15:45:44.356365 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.355943    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" event={"ID":"62e0dc58-e0f1-4071-baca-5fa054f1bd78","Type":"ContainerStarted","Data":"4aeaa7872324d52d855f883d557086eaf63e2de432e5b3e2052732c8d38bde91"}
Sep 08 15:45:44.373554 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373482    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nrh6\" (UniqueName: \"kubernetes.io/projected/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-kube-api-access-7nrh6\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.373554 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373551    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.373773 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373582    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.373773 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373613    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.373881 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373788    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.373881 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373828    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.373881 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373860    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-web-config\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373892    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-config-out\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373919    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373947    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.373994    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374018    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374064    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374067    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374134    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-config\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374166    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374207    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374239    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374292    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374647    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374790    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.374994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.374795    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.377779 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.377720    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.378414 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.378166    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.378795 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.378775    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.379572 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.379551    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.380846 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.380820    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.381884 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:45:44.381863    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-trusted-ca-bundle podName:9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a nodeName:}" failed. No retries permitted until 2026-09-08 15:45:44.880874334 +0000 UTC m=+54.377282283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-trusted-ca-bundle") pod "prometheus-k8s-0" (UID: "9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a") : configmap references non-existent config key: ca-bundle.crt
Sep 08 15:45:44.382707 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.382678    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.383038 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.383001    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.388096 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.387114    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.388096 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.387114    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.388096 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.387521    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-web-config\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.388096 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.388058    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.388846 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.388405    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.388846 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.388562    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-config-out\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.388846 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.388801    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.389075 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.388933    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-config\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.389375 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.389349    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nrh6\" (UniqueName: \"kubernetes.io/projected/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-kube-api-access-7nrh6\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.397352 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.395499    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.981369 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.981320    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:44.982195 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:44.982172    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:45.013211 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.013179    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:45.013383 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.013243    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:45.018921 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.018848    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:45.178924 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.178889    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:45.364947 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.364911    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" event={"ID":"62e0dc58-e0f1-4071-baca-5fa054f1bd78","Type":"ContainerStarted","Data":"c7bb033c0f293137ce6ac7790ca03bfe835fabadb694ab86ae026d6c33c291fd"}
Sep 08 15:45:45.365396 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.364992    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" event={"ID":"62e0dc58-e0f1-4071-baca-5fa054f1bd78","Type":"ContainerStarted","Data":"6d75d6bc664928901afaf5bdcc99b944d4f8089c4cf1a0f34829016ba25f0c27"}
Sep 08 15:45:45.365396 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.365059    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" event={"ID":"62e0dc58-e0f1-4071-baca-5fa054f1bd78","Type":"ContainerStarted","Data":"b5bd43eb447dc94359228e4e44811abdbde1127461fa10e42de74e75416dafd8"}
Sep 08 15:45:45.365396 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.365111    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:45.366574 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.366551    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv" event={"ID":"ff809900-2d1e-405a-be92-f1233a4daa58","Type":"ContainerStarted","Data":"25ba87b284ac974651a01da885e9f140246a0bb72319958894b2080a0753812d"}
Sep 08 15:45:45.370630 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.370610    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:45:45.390200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.389050    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774" podStartSLOduration=1.97817902 podStartE2EDuration="5.389017194s" totalImagesPullingTime="3.410838174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:45.385317556 +0000 UTC m=+54.881725525" watchObservedRunningTime="2026-09-08 15:45:45.389017194 +0000 UTC m=+54.885425161"
Sep 08 15:45:45.402381 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.402341    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv" podStartSLOduration=1.590148012 podStartE2EDuration="3.402328057s" totalImagesPullingTime="1.812180045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:45.401877671 +0000 UTC m=+54.898285631" watchObservedRunningTime="2026-09-08 15:45:45.402328057 +0000 UTC m=+54.898736023"
Sep 08 15:45:45.464578 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:45.464549    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:45:45.467322 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:45.467295    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eba29fc_a5b7_4ef3_9fb7_a77b43fa868a.slice/crio-1d62086897826f9704ef77e6cb6ab20be571c619705a24fedac8477b543165cf WatchSource:0}: Error finding container 1d62086897826f9704ef77e6cb6ab20be571c619705a24fedac8477b543165cf: Status 404 returned error can't find the container with id 1d62086897826f9704ef77e6cb6ab20be571c619705a24fedac8477b543165cf
Sep 08 15:45:46.294806 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:46.294776    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k9xkq"
Sep 08 15:45:46.371642 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:46.371610    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerStarted","Data":"1d62086897826f9704ef77e6cb6ab20be571c619705a24fedac8477b543165cf"}
Sep 08 15:45:47.377824 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:47.377792    2552 generic.go:356] "Generic (PLEG): container finished" podID="9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a" containerID="5adddb77c8d5febc239265d49fa391d618126ab6ef84bc6967a3bc40a78a25dd" exitCode=0
Sep 08 15:45:47.378333 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:47.377845    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerDied","Data":"5adddb77c8d5febc239265d49fa391d618126ab6ef84bc6967a3bc40a78a25dd"}
Sep 08 15:45:50.226238 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:50.226204    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ltprd"
Sep 08 15:45:51.377714 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.377631    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b4df7cffb-ql774"
Sep 08 15:45:51.398479 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.398451    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerStarted","Data":"89b49b9eb4c9c828382b69a54a463ee670066b5bea3105b123dc39b96d7b0a6e"}
Sep 08 15:45:51.398606 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.398507    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerStarted","Data":"9a031aa21213648b42cede486090a14a975bc4df6e5876e9f805a72b89b891bd"}
Sep 08 15:45:51.398606 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.398533    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerStarted","Data":"95bb9fb95e00819e50708292e25d690d210268e81dc62d089a350074e5354c96"}
Sep 08 15:45:51.398606 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.398557    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerStarted","Data":"43fb06b6c6dd856e8b7a245a8b38047e1d39df82788effb30e741a7977587230"}
Sep 08 15:45:51.398606 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.398581    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerStarted","Data":"1b72d83f307c730181293e9908321eafc3d2ff6f0dbff47de37de5c1a5ffa97e"}
Sep 08 15:45:51.398606 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.398604    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a","Type":"ContainerStarted","Data":"08fa96558d368cb49c980c0c33ef5b68b97f2ff94ae60ea31f95c45bbbe33e71"}
Sep 08 15:45:51.439447 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:51.439399    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.102647933 podStartE2EDuration="7.439382922s" totalImagesPullingTime="4.319468343s" totalInitContainerRuntime="17.266646ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:51.436902078 +0000 UTC m=+60.933310048" watchObservedRunningTime="2026-09-08 15:45:51.439382922 +0000 UTC m=+60.935790888"
Sep 08 15:45:55.179668 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:55.179630    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:45:56.706559 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.702984    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:56.706559 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.703117    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:56.707101 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.707082    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 15:45:56.707170 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.707153    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 15:45:56.715278 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.715250    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/37acb33a-d9a0-484e-94ed-69d21bbb302e-metrics-certs\") pod \"network-metrics-daemon-9jph2\" (UID: \"37acb33a-d9a0-484e-94ed-69d21bbb302e\") " pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:56.715435 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.715413    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80e81107-5b70-4123-87bf-8c0827d277d2-original-pull-secret\") pod \"global-pull-secret-syncer-wqb2w\" (UID: \"80e81107-5b70-4123-87bf-8c0827d277d2\") " pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:56.804128 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.804087    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:56.806901 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.806881    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 15:45:56.816811 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.816794    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 15:45:56.827099 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.827056    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f72d9\" (UniqueName: \"kubernetes.io/projected/0ab39284-a215-4381-b79a-faf6edfb5cb0-kube-api-access-f72d9\") pod \"network-check-target-dmf8l\" (UID: \"0ab39284-a215-4381-b79a-faf6edfb5cb0\") " pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:56.907442 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.907417    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xkdlq\""
Sep 08 15:45:56.911521 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.911508    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqb2w"
Sep 08 15:45:56.915652 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.915635    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:45:56.917721 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.917700    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-78hmt\""
Sep 08 15:45:56.925988 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:56.925945    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9jph2"
Sep 08 15:45:57.191915 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:57.191891    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dmf8l"]
Sep 08 15:45:57.192924 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:57.192904    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wqb2w"]
Sep 08 15:45:57.194588 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:57.194560    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e81107_5b70_4123_87bf_8c0827d277d2.slice/crio-7c3c77ce0af19ac97d16b9b59fdf42ee262f60f9efcf1254890e5a5c9f91d169 WatchSource:0}: Error finding container 7c3c77ce0af19ac97d16b9b59fdf42ee262f60f9efcf1254890e5a5c9f91d169: Status 404 returned error can't find the container with id 7c3c77ce0af19ac97d16b9b59fdf42ee262f60f9efcf1254890e5a5c9f91d169
Sep 08 15:45:57.195238 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:57.195209    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ab39284_a215_4381_b79a_faf6edfb5cb0.slice/crio-9cb87414e2a248910f93a19bbe5793d39495e8c2b29b078ef1c208b189082ae4 WatchSource:0}: Error finding container 9cb87414e2a248910f93a19bbe5793d39495e8c2b29b078ef1c208b189082ae4: Status 404 returned error can't find the container with id 9cb87414e2a248910f93a19bbe5793d39495e8c2b29b078ef1c208b189082ae4
Sep 08 15:45:57.213014 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:57.212986    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9jph2"]
Sep 08 15:45:57.216009 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:45:57.215980    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37acb33a_d9a0_484e_94ed_69d21bbb302e.slice/crio-43d4000dea98e44c4a3c4bbb1b5dbd7cc18a2cef6410a87bb5622eee3c6291a3 WatchSource:0}: Error finding container 43d4000dea98e44c4a3c4bbb1b5dbd7cc18a2cef6410a87bb5622eee3c6291a3: Status 404 returned error can't find the container with id 43d4000dea98e44c4a3c4bbb1b5dbd7cc18a2cef6410a87bb5622eee3c6291a3
Sep 08 15:45:57.422305 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:57.422269    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dmf8l" event={"ID":"0ab39284-a215-4381-b79a-faf6edfb5cb0","Type":"ContainerStarted","Data":"9cb87414e2a248910f93a19bbe5793d39495e8c2b29b078ef1c208b189082ae4"}
Sep 08 15:45:57.423444 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:57.423410    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jph2" event={"ID":"37acb33a-d9a0-484e-94ed-69d21bbb302e","Type":"ContainerStarted","Data":"43d4000dea98e44c4a3c4bbb1b5dbd7cc18a2cef6410a87bb5622eee3c6291a3"}
Sep 08 15:45:57.424571 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:57.424542    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wqb2w" event={"ID":"80e81107-5b70-4123-87bf-8c0827d277d2","Type":"ContainerStarted","Data":"7c3c77ce0af19ac97d16b9b59fdf42ee262f60f9efcf1254890e5a5c9f91d169"}
Sep 08 15:45:58.617607 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:58.617492    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ffc47769-shw9z"]
Sep 08 15:45:59.437432 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:59.437347    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jph2" event={"ID":"37acb33a-d9a0-484e-94ed-69d21bbb302e","Type":"ContainerStarted","Data":"ec40535253cf5839e20c64d7e2d4435ea78638d30b10caa4f61d4120a686626e"}
Sep 08 15:45:59.437432 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:59.437417    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9jph2" event={"ID":"37acb33a-d9a0-484e-94ed-69d21bbb302e","Type":"ContainerStarted","Data":"b1a0b982e7aa7c7f5e468a3b57cd6b7264929dc19ef3518c8007345df1c73ab2"}
Sep 08 15:45:59.455870 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:45:59.455808    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9jph2" podStartSLOduration=67.189569015 podStartE2EDuration="1m8.455784396s" totalImagesPullingTime="1.266215381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:59.453982227 +0000 UTC m=+68.950390197" watchObservedRunningTime="2026-09-08 15:45:59.455784396 +0000 UTC m=+68.952192364"
Sep 08 15:46:02.453446 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:02.453335    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dmf8l" event={"ID":"0ab39284-a215-4381-b79a-faf6edfb5cb0","Type":"ContainerStarted","Data":"adc42f1af42ca1af32f103cea27a3116ff0a06941d321f9c3b695a95db3511c5"}
Sep 08 15:46:02.453446 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:02.453422    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:46:02.455105 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:02.455076    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wqb2w" event={"ID":"80e81107-5b70-4123-87bf-8c0827d277d2","Type":"ContainerStarted","Data":"3449b4ccb2eb52cd454215ba0cfb86c125ceb72f8e0ba79bd317263791166ffa"}
Sep 08 15:46:02.470386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:02.470331    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dmf8l" podStartSLOduration=66.464644209 podStartE2EDuration="1m11.470315744s" totalImagesPullingTime="5.005671535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:02.469407698 +0000 UTC m=+71.965815666" watchObservedRunningTime="2026-09-08 15:46:02.470315744 +0000 UTC m=+71.966723701"
Sep 08 15:46:02.645921 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:02.645878    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:46:02.646142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:02.645959    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:46:10.275316 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:10.275268    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:46:10.332812 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:10.332787    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:46:10.362491 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:10.362440    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wqb2w" podStartSLOduration=74.345255842 podStartE2EDuration="1m19.362420565s" totalImagesPullingTime="5.017164723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:02.484080894 +0000 UTC m=+71.980488862" watchObservedRunningTime="2026-09-08 15:46:10.362420565 +0000 UTC m=+79.858828534"
Sep 08 15:46:10.501683 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:10.501660    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:46:22.651940 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:22.651814    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:46:22.655684 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:22.655661    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5487ccc788-sw5tv"
Sep 08 15:46:23.646518 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.646469    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7ffc47769-shw9z" podUID="7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" containerName="console" containerID="cri-o://4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc" gracePeriod=15
Sep 08 15:46:23.883723 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.883697    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:46:23.930791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.930722    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-config\") pod \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") "
Sep 08 15:46:23.930791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.930760    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-serving-cert\") pod \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") "
Sep 08 15:46:23.930791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.930788    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-service-ca\") pod \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") "
Sep 08 15:46:23.931016 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.930807    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-oauth-serving-cert\") pod \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") "
Sep 08 15:46:23.931016 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.930844    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkfvd\" (UniqueName: \"kubernetes.io/projected/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-kube-api-access-lkfvd\") pod \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") "
Sep 08 15:46:23.931016 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.930886    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-trusted-ca-bundle\") pod \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") "
Sep 08 15:46:23.931016 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.930911    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-oauth-config\") pod \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\" (UID: \"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6\") "
Sep 08 15:46:23.931246 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.931188    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-config" (OuterVolumeSpecName: "console-config") pod "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" (UID: "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:23.931308 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.931257    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" (UID: "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:23.931405 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.931262    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-service-ca" (OuterVolumeSpecName: "service-ca") pod "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" (UID: "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:23.931531 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.931500    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" (UID: "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:23.933301 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.933271    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" (UID: "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:46:23.933370 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.933331    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-kube-api-access-lkfvd" (OuterVolumeSpecName: "kube-api-access-lkfvd") pod "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" (UID: "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6"). InnerVolumeSpecName "kube-api-access-lkfvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:46:23.933370 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:23.933327    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" (UID: "7b6d08cd-35c6-461d-9ca1-87d22ab72fb6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:46:24.032003 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.031972    2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-trusted-ca-bundle\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:46:24.032003 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.031998    2552 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-oauth-config\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:46:24.032003 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.032009    2552 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-config\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:46:24.032245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.032018    2552 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-console-serving-cert\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:46:24.032245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.032049    2552 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:46:24.032245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.032057    2552 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-oauth-serving-cert\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:46:24.032245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.032065    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkfvd\" (UniqueName: \"kubernetes.io/projected/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6-kube-api-access-lkfvd\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:46:24.542756 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.542719    2552 generic.go:356] "Generic (PLEG): container finished" podID="7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" containerID="4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc" exitCode=0
Sep 08 15:46:24.542928 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.542765    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ffc47769-shw9z"
Sep 08 15:46:24.542928 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.542793    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffc47769-shw9z" event={"ID":"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6","Type":"ContainerDied","Data":"4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc"}
Sep 08 15:46:24.542928 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.542839    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ffc47769-shw9z" event={"ID":"7b6d08cd-35c6-461d-9ca1-87d22ab72fb6","Type":"ContainerDied","Data":"e94b1cdcab416c59126179b55fe2ea942a609f6c96aece6efb97a24069ca6d84"}
Sep 08 15:46:24.542928 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.542870    2552 scope.go:117] "RemoveContainer" containerID="4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc"
Sep 08 15:46:24.557277 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.557254    2552 scope.go:117] "RemoveContainer" containerID="4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc"
Sep 08 15:46:24.557609 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:46:24.557592    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc\": container with ID starting with 4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc not found: ID does not exist" containerID="4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc"
Sep 08 15:46:24.557668 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.557619    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc"} err="failed to get container status \"4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc\": rpc error: code = NotFound desc = could not find container \"4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc\": container with ID starting with 4c20dc8aa2d9aaf2691b4a157535d066852b1cac6ea4c0a322907ffa8c5ae1cc not found: ID does not exist"
Sep 08 15:46:24.567068 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.567047    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ffc47769-shw9z"]
Sep 08 15:46:24.569969 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:24.569948    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ffc47769-shw9z"]
Sep 08 15:46:25.099671 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:25.099636    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" path="/var/lib/kubelet/pods/7b6d08cd-35c6-461d-9ca1-87d22ab72fb6/volumes"
Sep 08 15:46:33.461482 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:46:33.461450    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dmf8l"
Sep 08 15:48:14.194188 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.194155    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b48nc"]
Sep 08 15:48:14.194629 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.194456    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" containerName="console"
Sep 08 15:48:14.194629 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.194470    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" containerName="console"
Sep 08 15:48:14.194629 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.194519    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b6d08cd-35c6-461d-9ca1-87d22ab72fb6" containerName="console"
Sep 08 15:48:14.196241 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.196223    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.198304 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.198282    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 08 15:48:14.198942 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.198922    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jxf2b\""
Sep 08 15:48:14.199175 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.199157    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 08 15:48:14.206743 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.206721    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b48nc"]
Sep 08 15:48:14.230054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.230012    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c584853-7287-457a-acef-9fc1c1448403-bound-sa-token\") pod \"cert-manager-55b4d954d5-b48nc\" (UID: \"8c584853-7287-457a-acef-9fc1c1448403\") " pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.230150 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.230089    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbq84\" (UniqueName: \"kubernetes.io/projected/8c584853-7287-457a-acef-9fc1c1448403-kube-api-access-mbq84\") pod \"cert-manager-55b4d954d5-b48nc\" (UID: \"8c584853-7287-457a-acef-9fc1c1448403\") " pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.331258 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.331231    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbq84\" (UniqueName: \"kubernetes.io/projected/8c584853-7287-457a-acef-9fc1c1448403-kube-api-access-mbq84\") pod \"cert-manager-55b4d954d5-b48nc\" (UID: \"8c584853-7287-457a-acef-9fc1c1448403\") " pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.331398 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.331283    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c584853-7287-457a-acef-9fc1c1448403-bound-sa-token\") pod \"cert-manager-55b4d954d5-b48nc\" (UID: \"8c584853-7287-457a-acef-9fc1c1448403\") " pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.342840 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.342811    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c584853-7287-457a-acef-9fc1c1448403-bound-sa-token\") pod \"cert-manager-55b4d954d5-b48nc\" (UID: \"8c584853-7287-457a-acef-9fc1c1448403\") " pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.343064 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.343042    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbq84\" (UniqueName: \"kubernetes.io/projected/8c584853-7287-457a-acef-9fc1c1448403-kube-api-access-mbq84\") pod \"cert-manager-55b4d954d5-b48nc\" (UID: \"8c584853-7287-457a-acef-9fc1c1448403\") " pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.519244 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.519146    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-b48nc"
Sep 08 15:48:14.785180 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.785084    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-b48nc"]
Sep 08 15:48:14.789564 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:48:14.789537    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c584853_7287_457a_acef_9fc1c1448403.slice/crio-e14b238b50e4526c6b2bc345f40a7ba062a79e481e78072a5b2d9b7590e6593b WatchSource:0}: Error finding container e14b238b50e4526c6b2bc345f40a7ba062a79e481e78072a5b2d9b7590e6593b: Status 404 returned error can't find the container with id e14b238b50e4526c6b2bc345f40a7ba062a79e481e78072a5b2d9b7590e6593b
Sep 08 15:48:14.945535 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:14.945499    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b48nc" event={"ID":"8c584853-7287-457a-acef-9fc1c1448403","Type":"ContainerStarted","Data":"e14b238b50e4526c6b2bc345f40a7ba062a79e481e78072a5b2d9b7590e6593b"}
Sep 08 15:48:18.402707 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.402671    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"]
Sep 08 15:48:18.405830 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.405809    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.408526 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.408505    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-d8s8z\""
Sep 08 15:48:18.409278 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.409262    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 15:48:18.409278 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.409275    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:48:18.417589 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.417569    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"]
Sep 08 15:48:18.467721 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.467695    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mzsx\" (UniqueName: \"kubernetes.io/projected/cd85a285-7ac6-4927-aa5e-b98535132f6d-kube-api-access-6mzsx\") pod \"openshift-lws-operator-fd8ccff4c-927zv\" (UID: \"cd85a285-7ac6-4927-aa5e-b98535132f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.467857 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.467729    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd85a285-7ac6-4927-aa5e-b98535132f6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-927zv\" (UID: \"cd85a285-7ac6-4927-aa5e-b98535132f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.568523 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.568481    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mzsx\" (UniqueName: \"kubernetes.io/projected/cd85a285-7ac6-4927-aa5e-b98535132f6d-kube-api-access-6mzsx\") pod \"openshift-lws-operator-fd8ccff4c-927zv\" (UID: \"cd85a285-7ac6-4927-aa5e-b98535132f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.568693 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.568532    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd85a285-7ac6-4927-aa5e-b98535132f6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-927zv\" (UID: \"cd85a285-7ac6-4927-aa5e-b98535132f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.568953 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.568931    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd85a285-7ac6-4927-aa5e-b98535132f6d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-927zv\" (UID: \"cd85a285-7ac6-4927-aa5e-b98535132f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.576477 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.576444    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mzsx\" (UniqueName: \"kubernetes.io/projected/cd85a285-7ac6-4927-aa5e-b98535132f6d-kube-api-access-6mzsx\") pod \"openshift-lws-operator-fd8ccff4c-927zv\" (UID: \"cd85a285-7ac6-4927-aa5e-b98535132f6d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.715178 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.715085    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"
Sep 08 15:48:18.967533 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.967450    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-b48nc" event={"ID":"8c584853-7287-457a-acef-9fc1c1448403","Type":"ContainerStarted","Data":"7a467297a4a166994bcec5b07c769a7b1f9df170639982eff901386b300afb6a"}
Sep 08 15:48:18.986445 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.986395    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-b48nc" podStartSLOduration=1.690926171 podStartE2EDuration="4.986377315s" totalImagesPullingTime="3.295451144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:48:18.984985091 +0000 UTC m=+208.481393070" watchObservedRunningTime="2026-09-08 15:48:18.986377315 +0000 UTC m=+208.482785282"
Sep 08 15:48:18.991884 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:18.991856    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv"]
Sep 08 15:48:18.994847 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:48:18.994819    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd85a285_7ac6_4927_aa5e_b98535132f6d.slice/crio-ce011b69b72899088f91fbe911b0ca1ffc73cf9b566666561e694d8cee0eeb09 WatchSource:0}: Error finding container ce011b69b72899088f91fbe911b0ca1ffc73cf9b566666561e694d8cee0eeb09: Status 404 returned error can't find the container with id ce011b69b72899088f91fbe911b0ca1ffc73cf9b566666561e694d8cee0eeb09
Sep 08 15:48:19.973807 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:19.973756    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv" event={"ID":"cd85a285-7ac6-4927-aa5e-b98535132f6d","Type":"ContainerStarted","Data":"ce011b69b72899088f91fbe911b0ca1ffc73cf9b566666561e694d8cee0eeb09"}
Sep 08 15:48:21.984195 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:21.984156    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv" event={"ID":"cd85a285-7ac6-4927-aa5e-b98535132f6d","Type":"ContainerStarted","Data":"12034f4cb39b03fe43fe3032aa48f278ec948582ecf9dd62c547e4f7bf834672"}
Sep 08 15:48:21.999068 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:21.999008    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-927zv" podStartSLOduration=1.440913322 podStartE2EDuration="3.998991729s" totalImagesPullingTime="2.558078407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:48:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:48:21.998574837 +0000 UTC m=+211.494982808" watchObservedRunningTime="2026-09-08 15:48:21.998991729 +0000 UTC m=+211.495399696"
Sep 08 15:48:41.289313 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.289277    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"]
Sep 08 15:48:41.292637 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.292621    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.295753 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.295724    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 15:48:41.295753 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.295742    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 15:48:41.295937 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.295825    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-89p79\""
Sep 08 15:48:41.295937 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.295859    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 15:48:41.296154 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.296137    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 15:48:41.303499 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.303474    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"]
Sep 08 15:48:41.455082 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.455048    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcd7edd-335e-4b49-b550-044bc1047133-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.455260 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.455095    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf8tn\" (UniqueName: \"kubernetes.io/projected/6bcd7edd-335e-4b49-b550-044bc1047133-kube-api-access-qf8tn\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.455260 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.455168    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bcd7edd-335e-4b49-b550-044bc1047133-tmp\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.455260 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.455197    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bcd7edd-335e-4b49-b550-044bc1047133-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.455415 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.455275    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bcd7edd-335e-4b49-b550-044bc1047133-manifests\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.556471 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.556368    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf8tn\" (UniqueName: \"kubernetes.io/projected/6bcd7edd-335e-4b49-b550-044bc1047133-kube-api-access-qf8tn\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.556471 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.556426    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bcd7edd-335e-4b49-b550-044bc1047133-tmp\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.556712 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.556550    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bcd7edd-335e-4b49-b550-044bc1047133-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.556712 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.556659    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bcd7edd-335e-4b49-b550-044bc1047133-manifests\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.556816 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.556797    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcd7edd-335e-4b49-b550-044bc1047133-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.557047 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.557008    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bcd7edd-335e-4b49-b550-044bc1047133-manifests\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.558681 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.558652    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bcd7edd-335e-4b49-b550-044bc1047133-tmp\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.558934 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.558917    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bcd7edd-335e-4b49-b550-044bc1047133-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.559010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.558990    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bcd7edd-335e-4b49-b550-044bc1047133-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.567326 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.567304    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf8tn\" (UniqueName: \"kubernetes.io/projected/6bcd7edd-335e-4b49-b550-044bc1047133-kube-api-access-qf8tn\") pod \"opendatahub-operator-controller-manager-5d4b5866d6-jfm4l\" (UID: \"6bcd7edd-335e-4b49-b550-044bc1047133\") " pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.602907 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.602876    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:41.886586 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:41.886521    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"]
Sep 08 15:48:41.888938 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:48:41.888913    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bcd7edd_335e_4b49_b550_044bc1047133.slice/crio-91cd688aff1a4a1ec86d8159e0a6db3a198ae68c8d813152385ec3b04116b1a6 WatchSource:0}: Error finding container 91cd688aff1a4a1ec86d8159e0a6db3a198ae68c8d813152385ec3b04116b1a6: Status 404 returned error can't find the container with id 91cd688aff1a4a1ec86d8159e0a6db3a198ae68c8d813152385ec3b04116b1a6
Sep 08 15:48:42.062850 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:42.062808    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l" event={"ID":"6bcd7edd-335e-4b49-b550-044bc1047133","Type":"ContainerStarted","Data":"91cd688aff1a4a1ec86d8159e0a6db3a198ae68c8d813152385ec3b04116b1a6"}
Sep 08 15:48:45.079498 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:45.079460    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l" event={"ID":"6bcd7edd-335e-4b49-b550-044bc1047133","Type":"ContainerStarted","Data":"fe923ac9cb1cf4f0f791291c45c41ab5d2a868d9eb0b0363f63612ee661259d9"}
Sep 08 15:48:46.085049 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:46.085000    2552 generic.go:356] "Generic (PLEG): container finished" podID="6bcd7edd-335e-4b49-b550-044bc1047133" containerID="fe923ac9cb1cf4f0f791291c45c41ab5d2a868d9eb0b0363f63612ee661259d9" exitCode=0
Sep 08 15:48:46.085555 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:46.085088    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l" event={"ID":"6bcd7edd-335e-4b49-b550-044bc1047133","Type":"ContainerDied","Data":"fe923ac9cb1cf4f0f791291c45c41ab5d2a868d9eb0b0363f63612ee661259d9"}
Sep 08 15:48:47.090881 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:47.090837    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l" event={"ID":"6bcd7edd-335e-4b49-b550-044bc1047133","Type":"ContainerStarted","Data":"91ef3f95d231ca94df12d8870f4c62d8d0bbf9b8cb3845e51d030a1fc4b542f2"}
Sep 08 15:48:47.091281 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:47.090979    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:48:47.118392 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:47.118343    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l" podStartSLOduration=1.729260295 podStartE2EDuration="6.118328242s" totalImagesPullingTime="3.349444567s" totalInitContainerRuntime="1.03962338s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:48:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:48:47.117637786 +0000 UTC m=+236.614045755" watchObservedRunningTime="2026-09-08 15:48:47.118328242 +0000 UTC m=+236.614736208"
Sep 08 15:48:58.097216 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:48:58.097181    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d4b5866d6-jfm4l"
Sep 08 15:49:06.603233 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.603196    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"]
Sep 08 15:49:06.613099 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.613074    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.615605 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.615580    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 15:49:06.615730 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.615601    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 15:49:06.616500 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.616310    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-v5zz6\""
Sep 08 15:49:06.616500 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.616381    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 15:49:06.616500 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.616409    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 15:49:06.619234 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.619208    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"]
Sep 08 15:49:06.662885 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.662857    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2697\" (UniqueName: \"kubernetes.io/projected/0e594c7f-af6f-432a-aec2-fd4399dbd462-kube-api-access-f2697\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.663042 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.662923    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e594c7f-af6f-432a-aec2-fd4399dbd462-tls-certs\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.663042 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.662953    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e594c7f-af6f-432a-aec2-fd4399dbd462-tmp\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.763906 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.763869    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2697\" (UniqueName: \"kubernetes.io/projected/0e594c7f-af6f-432a-aec2-fd4399dbd462-kube-api-access-f2697\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.764109 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.763944    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e594c7f-af6f-432a-aec2-fd4399dbd462-tls-certs\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.764109 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.763982    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e594c7f-af6f-432a-aec2-fd4399dbd462-tmp\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.764235 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:49:06.764103    2552 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 08 15:49:06.764235 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:49:06.764170    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e594c7f-af6f-432a-aec2-fd4399dbd462-tls-certs podName:0e594c7f-af6f-432a-aec2-fd4399dbd462 nodeName:}" failed. No retries permitted until 2026-09-08 15:49:07.264154513 +0000 UTC m=+256.760562462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0e594c7f-af6f-432a-aec2-fd4399dbd462-tls-certs") pod "kube-auth-proxy-74bfc7b5fc-725tn" (UID: "0e594c7f-af6f-432a-aec2-fd4399dbd462") : secret "kube-auth-proxy-tls" not found
Sep 08 15:49:06.766352 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.766324    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e594c7f-af6f-432a-aec2-fd4399dbd462-tmp\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:06.772746 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:06.772720    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2697\" (UniqueName: \"kubernetes.io/projected/0e594c7f-af6f-432a-aec2-fd4399dbd462-kube-api-access-f2697\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:07.268304 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:07.268268    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e594c7f-af6f-432a-aec2-fd4399dbd462-tls-certs\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:07.270736 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:07.270710    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e594c7f-af6f-432a-aec2-fd4399dbd462-tls-certs\") pod \"kube-auth-proxy-74bfc7b5fc-725tn\" (UID: \"0e594c7f-af6f-432a-aec2-fd4399dbd462\") " pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:07.524012 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:07.523915    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"
Sep 08 15:49:07.793432 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:07.793351    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn"]
Sep 08 15:49:07.796473 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:49:07.796446    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e594c7f_af6f_432a_aec2_fd4399dbd462.slice/crio-edac2fc01d22212af292cb35d3036c13a80a36a9dadcb0c2da370b57fa98212a WatchSource:0}: Error finding container edac2fc01d22212af292cb35d3036c13a80a36a9dadcb0c2da370b57fa98212a: Status 404 returned error can't find the container with id edac2fc01d22212af292cb35d3036c13a80a36a9dadcb0c2da370b57fa98212a
Sep 08 15:49:08.175850 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:08.175813    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn" event={"ID":"0e594c7f-af6f-432a-aec2-fd4399dbd462","Type":"ContainerStarted","Data":"edac2fc01d22212af292cb35d3036c13a80a36a9dadcb0c2da370b57fa98212a"}
Sep 08 15:49:11.191088 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:11.191021    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn" event={"ID":"0e594c7f-af6f-432a-aec2-fd4399dbd462","Type":"ContainerStarted","Data":"f709aad352aa8d69c6555ce5d62c5cda20c5fdd55fc04c057864b3b5666f62eb"}
Sep 08 15:49:11.208472 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:11.208426    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74bfc7b5fc-725tn" podStartSLOduration=2.498804203 podStartE2EDuration="5.208410276s" totalImagesPullingTime="2.709606073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:11.207110492 +0000 UTC m=+260.703518458" watchObservedRunningTime="2026-09-08 15:49:11.208410276 +0000 UTC m=+260.704818242"
Sep 08 15:49:12.003789 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.003755    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hlpsc"]
Sep 08 15:49:12.006385 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.006370    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.008689 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.008662    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-67wr9\""
Sep 08 15:49:12.008878 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.008860    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 08 15:49:12.019372 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.019339    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hlpsc"]
Sep 08 15:49:12.114339 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.114308    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/992f3f5a-d514-441a-b69c-d44a7d9d2311-cert\") pod \"odh-model-controller-7998c67dc4-hlpsc\" (UID: \"992f3f5a-d514-441a-b69c-d44a7d9d2311\") " pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.114566 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.114535    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjq99\" (UniqueName: \"kubernetes.io/projected/992f3f5a-d514-441a-b69c-d44a7d9d2311-kube-api-access-rjq99\") pod \"odh-model-controller-7998c67dc4-hlpsc\" (UID: \"992f3f5a-d514-441a-b69c-d44a7d9d2311\") " pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.215985 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.215949    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/992f3f5a-d514-441a-b69c-d44a7d9d2311-cert\") pod \"odh-model-controller-7998c67dc4-hlpsc\" (UID: \"992f3f5a-d514-441a-b69c-d44a7d9d2311\") " pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.216380 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.216018    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjq99\" (UniqueName: \"kubernetes.io/projected/992f3f5a-d514-441a-b69c-d44a7d9d2311-kube-api-access-rjq99\") pod \"odh-model-controller-7998c67dc4-hlpsc\" (UID: \"992f3f5a-d514-441a-b69c-d44a7d9d2311\") " pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.218368 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.218340    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/992f3f5a-d514-441a-b69c-d44a7d9d2311-cert\") pod \"odh-model-controller-7998c67dc4-hlpsc\" (UID: \"992f3f5a-d514-441a-b69c-d44a7d9d2311\") " pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.224452 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.224428    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjq99\" (UniqueName: \"kubernetes.io/projected/992f3f5a-d514-441a-b69c-d44a7d9d2311-kube-api-access-rjq99\") pod \"odh-model-controller-7998c67dc4-hlpsc\" (UID: \"992f3f5a-d514-441a-b69c-d44a7d9d2311\") " pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.319566 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.319475    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:12.455551 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.455521    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nrzlr"]
Sep 08 15:49:12.458221 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.458204    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:12.459986 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.459955    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 08 15:49:12.469130 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.469092    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nrzlr"]
Sep 08 15:49:12.518104 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.518076    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b038f1d-af2d-4c8f-990a-019779b860bc-tls-certs\") pod \"model-serving-api-54d89bd5fc-nrzlr\" (UID: \"8b038f1d-af2d-4c8f-990a-019779b860bc\") " pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:12.518243 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.518119    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8fhv\" (UniqueName: \"kubernetes.io/projected/8b038f1d-af2d-4c8f-990a-019779b860bc-kube-api-access-l8fhv\") pod \"model-serving-api-54d89bd5fc-nrzlr\" (UID: \"8b038f1d-af2d-4c8f-990a-019779b860bc\") " pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:12.591262 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.591177    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hlpsc"]
Sep 08 15:49:12.593414 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:49:12.593386    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992f3f5a_d514_441a_b69c_d44a7d9d2311.slice/crio-4ea11a9c377417103b700771d30839fcb7223f1cd5e5c034a8935b4a45237767 WatchSource:0}: Error finding container 4ea11a9c377417103b700771d30839fcb7223f1cd5e5c034a8935b4a45237767: Status 404 returned error can't find the container with id 4ea11a9c377417103b700771d30839fcb7223f1cd5e5c034a8935b4a45237767
Sep 08 15:49:12.619271 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.619246    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b038f1d-af2d-4c8f-990a-019779b860bc-tls-certs\") pod \"model-serving-api-54d89bd5fc-nrzlr\" (UID: \"8b038f1d-af2d-4c8f-990a-019779b860bc\") " pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:12.619377 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.619286    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8fhv\" (UniqueName: \"kubernetes.io/projected/8b038f1d-af2d-4c8f-990a-019779b860bc-kube-api-access-l8fhv\") pod \"model-serving-api-54d89bd5fc-nrzlr\" (UID: \"8b038f1d-af2d-4c8f-990a-019779b860bc\") " pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:12.619421 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:49:12.619400    2552 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 08 15:49:12.619472 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:49:12.619461    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b038f1d-af2d-4c8f-990a-019779b860bc-tls-certs podName:8b038f1d-af2d-4c8f-990a-019779b860bc nodeName:}" failed. No retries permitted until 2026-09-08 15:49:13.119441208 +0000 UTC m=+262.615849153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8b038f1d-af2d-4c8f-990a-019779b860bc-tls-certs") pod "model-serving-api-54d89bd5fc-nrzlr" (UID: "8b038f1d-af2d-4c8f-990a-019779b860bc") : secret "model-serving-api-tls" not found
Sep 08 15:49:12.627304 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:12.627281    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8fhv\" (UniqueName: \"kubernetes.io/projected/8b038f1d-af2d-4c8f-990a-019779b860bc-kube-api-access-l8fhv\") pod \"model-serving-api-54d89bd5fc-nrzlr\" (UID: \"8b038f1d-af2d-4c8f-990a-019779b860bc\") " pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:13.123332 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.123301    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b038f1d-af2d-4c8f-990a-019779b860bc-tls-certs\") pod \"model-serving-api-54d89bd5fc-nrzlr\" (UID: \"8b038f1d-af2d-4c8f-990a-019779b860bc\") " pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:13.125690 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.125663    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b038f1d-af2d-4c8f-990a-019779b860bc-tls-certs\") pod \"model-serving-api-54d89bd5fc-nrzlr\" (UID: \"8b038f1d-af2d-4c8f-990a-019779b860bc\") " pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:13.200204 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.200131    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc" event={"ID":"992f3f5a-d514-441a-b69c-d44a7d9d2311","Type":"ContainerStarted","Data":"4ea11a9c377417103b700771d30839fcb7223f1cd5e5c034a8935b4a45237767"}
Sep 08 15:49:13.369487 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.369181    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:13.659623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.654953    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nrzlr"]
Sep 08 15:49:13.659623 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.657303    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"]
Sep 08 15:49:13.661469 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.661443    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.664113 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.663986    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-clm5q\""
Sep 08 15:49:13.664701 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.664328    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 15:49:13.677637 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.677515    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"]
Sep 08 15:49:13.730304 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730270    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730476 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730323    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730476 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730365    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be0b2394-e834-406c-a95f-c06bd741594a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730476 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730444    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be0b2394-e834-406c-a95f-c06bd741594a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730648 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730500    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730648 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730557    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730751 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730644    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730751 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730683    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2lsn\" (UniqueName: \"kubernetes.io/projected/be0b2394-e834-406c-a95f-c06bd741594a-kube-api-access-t2lsn\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.730751 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.730731    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be0b2394-e834-406c-a95f-c06bd741594a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.832402 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832311    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.832848 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832652    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.832848 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832741    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.832848 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832776    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2lsn\" (UniqueName: \"kubernetes.io/projected/be0b2394-e834-406c-a95f-c06bd741594a-kube-api-access-t2lsn\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.832848 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832809    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.832848 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832822    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be0b2394-e834-406c-a95f-c06bd741594a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.833279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832881    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.833279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832917    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.833279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.832961    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be0b2394-e834-406c-a95f-c06bd741594a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.833279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.833006    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be0b2394-e834-406c-a95f-c06bd741594a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.833279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.833018    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.834171 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.833402    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.834171 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.833427    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.834171 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.834014    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be0b2394-e834-406c-a95f-c06bd741594a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.836762 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.836697    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be0b2394-e834-406c-a95f-c06bd741594a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.837676 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.837610    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be0b2394-e834-406c-a95f-c06bd741594a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.841272 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.841239    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be0b2394-e834-406c-a95f-c06bd741594a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.843534 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.843489    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2lsn\" (UniqueName: \"kubernetes.io/projected/be0b2394-e834-406c-a95f-c06bd741594a-kube-api-access-t2lsn\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh\" (UID: \"be0b2394-e834-406c-a95f-c06bd741594a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:13.986158 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:13.986076    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:14.207812 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:14.207752    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr" event={"ID":"8b038f1d-af2d-4c8f-990a-019779b860bc","Type":"ContainerStarted","Data":"b739b2cf28a55e8bd8890d3557eb6880ff32cc99f16cd96f8c388fb796df3777"}
Sep 08 15:49:14.277726 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:14.277699    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"]
Sep 08 15:49:15.719539 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:49:15.719506    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0b2394_e834_406c_a95f_c06bd741594a.slice/crio-a605305958395dc226cc7c15cc0c5159e0cacba3f1e6111b5996e381f0441020 WatchSource:0}: Error finding container a605305958395dc226cc7c15cc0c5159e0cacba3f1e6111b5996e381f0441020: Status 404 returned error can't find the container with id a605305958395dc226cc7c15cc0c5159e0cacba3f1e6111b5996e381f0441020
Sep 08 15:49:16.219236 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:16.219194    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc" event={"ID":"992f3f5a-d514-441a-b69c-d44a7d9d2311","Type":"ContainerStarted","Data":"9b9a7bd816652bba8f2614c607b93f7b5a8448e28cce7eaa4170b169dd9fac89"}
Sep 08 15:49:16.219414 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:16.219306    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:16.220908 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:16.220877    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr" event={"ID":"8b038f1d-af2d-4c8f-990a-019779b860bc","Type":"ContainerStarted","Data":"f9e34186623e7f319c9e4e42af13912e49d508568c134c033ed08c8772f8c808"}
Sep 08 15:49:16.221050 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:16.220956    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:16.222199 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:16.222178    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh" event={"ID":"be0b2394-e834-406c-a95f-c06bd741594a","Type":"ContainerStarted","Data":"a605305958395dc226cc7c15cc0c5159e0cacba3f1e6111b5996e381f0441020"}
Sep 08 15:49:16.236622 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:16.236575    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc" podStartSLOduration=1.7036917059999999 podStartE2EDuration="5.236560818s" totalImagesPullingTime="3.532869112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:16.235337728 +0000 UTC m=+265.731745698" watchObservedRunningTime="2026-09-08 15:49:16.236560818 +0000 UTC m=+265.732968785"
Sep 08 15:49:16.253740 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:16.253686    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr" podStartSLOduration=1.7994284980000002 podStartE2EDuration="4.253666074s" totalImagesPullingTime="2.454237576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:16.253217205 +0000 UTC m=+265.749625173" watchObservedRunningTime="2026-09-08 15:49:16.253666074 +0000 UTC m=+265.750074039"
Sep 08 15:49:18.509294 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.509254    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"]
Sep 08 15:49:18.516613 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.516588    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.520200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.519257    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 08 15:49:18.520200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.519944    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 08 15:49:18.520200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.519966    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 08 15:49:18.520200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.519948    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lpf5t\""
Sep 08 15:49:18.520483 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.520464    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"]
Sep 08 15:49:18.582015 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.581974    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/538aa371-7e65-4262-81e2-05b425b4c310-cert\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.582203 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.582045    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cp8d\" (UniqueName: \"kubernetes.io/projected/538aa371-7e65-4262-81e2-05b425b4c310-kube-api-access-4cp8d\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.582203 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.582120    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/538aa371-7e65-4262-81e2-05b425b4c310-metrics-cert\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.582203 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.582156    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/538aa371-7e65-4262-81e2-05b425b4c310-manager-config\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.651103 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.651058    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:49:18.651206 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.651156    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:49:18.651206 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.651195    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:49:18.683136 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.683101    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/538aa371-7e65-4262-81e2-05b425b4c310-metrics-cert\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.683257 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.683167    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/538aa371-7e65-4262-81e2-05b425b4c310-manager-config\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.683322 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.683276    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/538aa371-7e65-4262-81e2-05b425b4c310-cert\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.683322 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.683312    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cp8d\" (UniqueName: \"kubernetes.io/projected/538aa371-7e65-4262-81e2-05b425b4c310-kube-api-access-4cp8d\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.683794 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.683769    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/538aa371-7e65-4262-81e2-05b425b4c310-manager-config\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.685476 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.685444    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/538aa371-7e65-4262-81e2-05b425b4c310-metrics-cert\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.685593 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.685576    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/538aa371-7e65-4262-81e2-05b425b4c310-cert\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.691264 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.691241    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cp8d\" (UniqueName: \"kubernetes.io/projected/538aa371-7e65-4262-81e2-05b425b4c310-kube-api-access-4cp8d\") pod \"lws-controller-manager-578d6cf975-lwzlv\" (UID: \"538aa371-7e65-4262-81e2-05b425b4c310\") " pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:18.830200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:18.830111    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:19.152889 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:49:19.152856    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538aa371_7e65_4262_81e2_05b425b4c310.slice/crio-37ee8da45787e571141a9c33a9086501b1d83f4fa97b88affd4baecf689815da WatchSource:0}: Error finding container 37ee8da45787e571141a9c33a9086501b1d83f4fa97b88affd4baecf689815da: Status 404 returned error can't find the container with id 37ee8da45787e571141a9c33a9086501b1d83f4fa97b88affd4baecf689815da
Sep 08 15:49:19.153682 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:19.153628    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"]
Sep 08 15:49:19.239660 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:19.239618    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh" event={"ID":"be0b2394-e834-406c-a95f-c06bd741594a","Type":"ContainerStarted","Data":"3b858415d7040442f0bae650317ad76d0e07f35b9348a68149b2f689d26de54c"}
Sep 08 15:49:19.240880 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:19.240855    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv" event={"ID":"538aa371-7e65-4262-81e2-05b425b4c310","Type":"ContainerStarted","Data":"37ee8da45787e571141a9c33a9086501b1d83f4fa97b88affd4baecf689815da"}
Sep 08 15:49:19.264681 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:19.264629    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh" podStartSLOduration=3.335488951 podStartE2EDuration="6.264615402s" totalImagesPullingTime="2.929126451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:19.262709588 +0000 UTC m=+268.759117581" watchObservedRunningTime="2026-09-08 15:49:19.264615402 +0000 UTC m=+268.761023369"
Sep 08 15:49:19.986467 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:19.986425    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:19.992887 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:19.992835    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:20.247141 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:20.247070    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:20.248251 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:20.248228    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh"
Sep 08 15:49:21.252275 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:21.252237    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv" event={"ID":"538aa371-7e65-4262-81e2-05b425b4c310","Type":"ContainerStarted","Data":"44424e77ef6aec2a4cc1d15dd774b12f81f8985374036aabb817deffb1cfcd6d"}
Sep 08 15:49:21.272446 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:21.272394    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv" podStartSLOduration=1.740577943 podStartE2EDuration="3.272376427s" totalImagesPullingTime="1.531798484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:21.27170212 +0000 UTC m=+270.768110104" watchObservedRunningTime="2026-09-08 15:49:21.272376427 +0000 UTC m=+270.768784393"
Sep 08 15:49:22.256701 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:22.256669    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:27.231129 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:27.231100    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hlpsc"
Sep 08 15:49:27.233081 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:27.233062    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nrzlr"
Sep 08 15:49:33.263946 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:33.263909    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-578d6cf975-lwzlv"
Sep 08 15:49:50.869566 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:49:50.869536    2552 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 15:50:58.053852 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.053819    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"]
Sep 08 15:50:58.056100 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.056085    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.058433 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.058411    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 08 15:50:58.058584 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.058503    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9sb96\""
Sep 08 15:50:58.058584 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.058518    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 08 15:50:58.059430 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.059406    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 15:50:58.059559 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.059548    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 15:50:58.064929 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.064910    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"]
Sep 08 15:50:58.152502 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.152465    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m49d\" (UniqueName: \"kubernetes.io/projected/a91bab1c-3256-476e-875c-67aa5d835154-kube-api-access-6m49d\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.152663 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.152554    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a91bab1c-3256-476e-875c-67aa5d835154-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.152752 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.152733    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.253535 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.253500    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.253687 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.253548    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m49d\" (UniqueName: \"kubernetes.io/projected/a91bab1c-3256-476e-875c-67aa5d835154-kube-api-access-6m49d\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.253687 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.253586    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a91bab1c-3256-476e-875c-67aa5d835154-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.253687 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:50:58.253649    2552 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 08 15:50:58.253804 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:50:58.253734    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert podName:a91bab1c-3256-476e-875c-67aa5d835154 nodeName:}" failed. No retries permitted until 2026-09-08 15:50:58.753712488 +0000 UTC m=+368.250120441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-rbf7d" (UID: "a91bab1c-3256-476e-875c-67aa5d835154") : secret "plugin-serving-cert" not found
Sep 08 15:50:58.254218 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.254200    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a91bab1c-3256-476e-875c-67aa5d835154-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.262069 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.262040    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m49d\" (UniqueName: \"kubernetes.io/projected/a91bab1c-3256-476e-875c-67aa5d835154-kube-api-access-6m49d\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.757653 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.757621    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.760008 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.759982    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-rbf7d\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:58.965731 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:58.965696    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:50:59.234566 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:59.234541    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"]
Sep 08 15:50:59.236792 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:50:59.236763    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda91bab1c_3256_476e_875c_67aa5d835154.slice/crio-f26d7e16c6df744ebf5af98c631ce9d787ec5e5dd6610d7687ab1c0da76aaeba WatchSource:0}: Error finding container f26d7e16c6df744ebf5af98c631ce9d787ec5e5dd6610d7687ab1c0da76aaeba: Status 404 returned error can't find the container with id f26d7e16c6df744ebf5af98c631ce9d787ec5e5dd6610d7687ab1c0da76aaeba
Sep 08 15:50:59.238274 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:59.238251    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:50:59.682700 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:50:59.682665    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d" event={"ID":"a91bab1c-3256-476e-875c-67aa5d835154","Type":"ContainerStarted","Data":"f26d7e16c6df744ebf5af98c631ce9d787ec5e5dd6610d7687ab1c0da76aaeba"}
Sep 08 15:51:05.712748 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:05.712709    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d" event={"ID":"a91bab1c-3256-476e-875c-67aa5d835154","Type":"ContainerStarted","Data":"a11c3c33acb4ae696c7b67db835e26bc4c61971e501d51ae8675260d74647902"}
Sep 08 15:51:05.730019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:05.729968    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d" podStartSLOduration=2.2305806280000002 podStartE2EDuration="7.729952813s" totalImagesPullingTime="5.499372185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:50:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:05.728757891 +0000 UTC m=+375.225165860" watchObservedRunningTime="2026-09-08 15:51:05.729952813 +0000 UTC m=+375.226360780"
Sep 08 15:51:37.642218 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.642185    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"]
Sep 08 15:51:37.659819 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.659783    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"]
Sep 08 15:51:37.659967 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.659912    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.662095 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.662074    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ltwln\""
Sep 08 15:51:37.686009 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.685983    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686013    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686055    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686142 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686121    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686151    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686187    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686210    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686245 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686238    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnnqb\" (UniqueName: \"kubernetes.io/projected/e8c13238-ca5b-4de2-a21f-4f187ca413b6-kube-api-access-qnnqb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.686366 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.686279    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.787604 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787571    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.787768 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787634    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.787768 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787651    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.787768 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787673    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.787768 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787694    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.787768 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787712    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788020 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787885    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788020 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787928    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788020 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.787966    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnnqb\" (UniqueName: \"kubernetes.io/projected/e8c13238-ca5b-4de2-a21f-4f187ca413b6-kube-api-access-qnnqb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788242 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.788178    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788242 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.788215    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788345 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.788288    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788481 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.788457    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.788601 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.788581    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.789932 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.789912    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.790301 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.790281    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.800864 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.800840    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnnqb\" (UniqueName: \"kubernetes.io/projected/e8c13238-ca5b-4de2-a21f-4f187ca413b6-kube-api-access-qnnqb\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.802653 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.802636    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8c13238-ca5b-4de2-a21f-4f187ca413b6-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ksf69\" (UID: \"e8c13238-ca5b-4de2-a21f-4f187ca413b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:37.971076 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:37.970976    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:38.243063 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.243020    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"]
Sep 08 15:51:38.245263 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:51:38.245225    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c13238_ca5b_4de2_a21f_4f187ca413b6.slice/crio-568f2b8db0c0ccc38c5c7b8c6bf9567b415bfaa4986abb137dcca8861f93138f WatchSource:0}: Error finding container 568f2b8db0c0ccc38c5c7b8c6bf9567b415bfaa4986abb137dcca8861f93138f: Status 404 returned error can't find the container with id 568f2b8db0c0ccc38c5c7b8c6bf9567b415bfaa4986abb137dcca8861f93138f
Sep 08 15:51:38.247264 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.247232    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:38.247351 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.247307    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:38.247422 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.247367    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:38.860484 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.860449    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" event={"ID":"e8c13238-ca5b-4de2-a21f-4f187ca413b6","Type":"ContainerStarted","Data":"2df33cc801b255fd77c9119b31c6d774f94a833abdd578da57a9783d30098c62"}
Sep 08 15:51:38.860874 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.860510    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" event={"ID":"e8c13238-ca5b-4de2-a21f-4f187ca413b6","Type":"ContainerStarted","Data":"568f2b8db0c0ccc38c5c7b8c6bf9567b415bfaa4986abb137dcca8861f93138f"}
Sep 08 15:51:38.880845 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.880796    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" podStartSLOduration=1.880778421 podStartE2EDuration="1.880778421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:38.879642293 +0000 UTC m=+408.376050286" watchObservedRunningTime="2026-09-08 15:51:38.880778421 +0000 UTC m=+408.377186434"
Sep 08 15:51:38.971287 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:38.971253    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:39.976100 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:39.976071    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:40.870519 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:40.870491    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:40.871452 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:40.871430    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:51:52.201605 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.201572    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-nsbfn"]
Sep 08 15:51:52.203984 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.203962    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-nsbfn"
Sep 08 15:51:52.206010 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.205985    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2znf2\""
Sep 08 15:51:52.212202 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.212176    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-nsbfn"]
Sep 08 15:51:52.310755 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.310721    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45sdp\" (UniqueName: \"kubernetes.io/projected/5be9962e-877b-43da-822b-0d89dc2a203d-kube-api-access-45sdp\") pod \"authorino-c548b5d7d-nsbfn\" (UID: \"5be9962e-877b-43da-822b-0d89dc2a203d\") " pod="kuadrant-system/authorino-c548b5d7d-nsbfn"
Sep 08 15:51:52.412215 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.412164    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45sdp\" (UniqueName: \"kubernetes.io/projected/5be9962e-877b-43da-822b-0d89dc2a203d-kube-api-access-45sdp\") pod \"authorino-c548b5d7d-nsbfn\" (UID: \"5be9962e-877b-43da-822b-0d89dc2a203d\") " pod="kuadrant-system/authorino-c548b5d7d-nsbfn"
Sep 08 15:51:52.420031 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.420007    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45sdp\" (UniqueName: \"kubernetes.io/projected/5be9962e-877b-43da-822b-0d89dc2a203d-kube-api-access-45sdp\") pod \"authorino-c548b5d7d-nsbfn\" (UID: \"5be9962e-877b-43da-822b-0d89dc2a203d\") " pod="kuadrant-system/authorino-c548b5d7d-nsbfn"
Sep 08 15:51:52.513348 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.513269    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-nsbfn"
Sep 08 15:51:52.781488 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.781464    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-nsbfn"]
Sep 08 15:51:52.783768 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:51:52.783739    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5be9962e_877b_43da_822b_0d89dc2a203d.slice/crio-784e14aaeae27bcbb250783acce5f2b06939cc80506b044917ab921a14ea0d40 WatchSource:0}: Error finding container 784e14aaeae27bcbb250783acce5f2b06939cc80506b044917ab921a14ea0d40: Status 404 returned error can't find the container with id 784e14aaeae27bcbb250783acce5f2b06939cc80506b044917ab921a14ea0d40
Sep 08 15:51:52.923936 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:51:52.923903    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-nsbfn" event={"ID":"5be9962e-877b-43da-822b-0d89dc2a203d","Type":"ContainerStarted","Data":"784e14aaeae27bcbb250783acce5f2b06939cc80506b044917ab921a14ea0d40"}
Sep 08 15:52:05.982829 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:05.982788    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-nsbfn" event={"ID":"5be9962e-877b-43da-822b-0d89dc2a203d","Type":"ContainerStarted","Data":"7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b"}
Sep 08 15:52:06.003601 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:06.003535    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-nsbfn" podStartSLOduration=1.801992372 podStartE2EDuration="14.003512809s" totalImagesPullingTime="12.201520437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:52:06.002088483 +0000 UTC m=+435.498496450" watchObservedRunningTime="2026-09-08 15:52:06.003512809 +0000 UTC m=+435.499920777"
Sep 08 15:52:21.173434 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:21.173349    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-nsbfn"]
Sep 08 15:52:21.173975 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:21.173743    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-nsbfn" podUID="5be9962e-877b-43da-822b-0d89dc2a203d" containerName="authorino" containerID="cri-o://7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b" gracePeriod=30
Sep 08 15:52:21.419328 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:21.419304    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-nsbfn"
Sep 08 15:52:21.580567 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:21.580484    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45sdp\" (UniqueName: \"kubernetes.io/projected/5be9962e-877b-43da-822b-0d89dc2a203d-kube-api-access-45sdp\") pod \"5be9962e-877b-43da-822b-0d89dc2a203d\" (UID: \"5be9962e-877b-43da-822b-0d89dc2a203d\") "
Sep 08 15:52:21.582594 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:21.582566    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be9962e-877b-43da-822b-0d89dc2a203d-kube-api-access-45sdp" (OuterVolumeSpecName: "kube-api-access-45sdp") pod "5be9962e-877b-43da-822b-0d89dc2a203d" (UID: "5be9962e-877b-43da-822b-0d89dc2a203d"). InnerVolumeSpecName "kube-api-access-45sdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:52:21.681946 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:21.681912    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45sdp\" (UniqueName: \"kubernetes.io/projected/5be9962e-877b-43da-822b-0d89dc2a203d-kube-api-access-45sdp\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:52:22.055389 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.055345    2552 generic.go:356] "Generic (PLEG): container finished" podID="5be9962e-877b-43da-822b-0d89dc2a203d" containerID="7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b" exitCode=0
Sep 08 15:52:22.055389 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.055390    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-nsbfn"
Sep 08 15:52:22.055645 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.055397    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-nsbfn" event={"ID":"5be9962e-877b-43da-822b-0d89dc2a203d","Type":"ContainerDied","Data":"7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b"}
Sep 08 15:52:22.055645 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.055438    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-nsbfn" event={"ID":"5be9962e-877b-43da-822b-0d89dc2a203d","Type":"ContainerDied","Data":"784e14aaeae27bcbb250783acce5f2b06939cc80506b044917ab921a14ea0d40"}
Sep 08 15:52:22.055645 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.055463    2552 scope.go:117] "RemoveContainer" containerID="7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b"
Sep 08 15:52:22.064574 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.064556    2552 scope.go:117] "RemoveContainer" containerID="7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b"
Sep 08 15:52:22.064863 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:52:22.064845    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b\": container with ID starting with 7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b not found: ID does not exist" containerID="7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b"
Sep 08 15:52:22.064906 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.064873    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b"} err="failed to get container status \"7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b\": rpc error: code = NotFound desc = could not find container \"7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b\": container with ID starting with 7272fc48d9ce290681328aaf494bf1feb368ba4e1f57a565259e4182a7ff9d6b not found: ID does not exist"
Sep 08 15:52:22.075565 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.075544    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-nsbfn"]
Sep 08 15:52:22.079286 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:22.079265    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-nsbfn"]
Sep 08 15:52:23.100650 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:23.100618    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be9962e-877b-43da-822b-0d89dc2a203d" path="/var/lib/kubelet/pods/5be9962e-877b-43da-822b-0d89dc2a203d/volumes"
Sep 08 15:52:26.190204 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.190168    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75db7f7669-9w88x"]
Sep 08 15:52:26.190595 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.190540    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be9962e-877b-43da-822b-0d89dc2a203d" containerName="authorino"
Sep 08 15:52:26.190595 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.190553    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be9962e-877b-43da-822b-0d89dc2a203d" containerName="authorino"
Sep 08 15:52:26.190668 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.190609    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be9962e-877b-43da-822b-0d89dc2a203d" containerName="authorino"
Sep 08 15:52:26.194241 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.194220    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.196360 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.196338    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 15:52:26.197326 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.197308    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 15:52:26.197429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.197350    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 15:52:26.197429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.197414    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4tklt\""
Sep 08 15:52:26.207660 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.207636    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75db7f7669-9w88x"]
Sep 08 15:52:26.223750 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.223729    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45fpk\" (UniqueName: \"kubernetes.io/projected/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-kube-api-access-45fpk\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.223847 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.223784    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-webhook-cert\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.223847 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.223802    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-metrics-certs\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.325178 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.325140    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-webhook-cert\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.325178 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.325176    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-metrics-certs\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.325434 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.325245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45fpk\" (UniqueName: \"kubernetes.io/projected/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-kube-api-access-45fpk\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.327735 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.327708    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-webhook-cert\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.327852 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.327829    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-metrics-certs\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.333291 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.333262    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45fpk\" (UniqueName: \"kubernetes.io/projected/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-kube-api-access-45fpk\") pod \"maas-controller-75db7f7669-9w88x\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") " pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.504195 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.504112    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:26.776905 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:26.776879    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75db7f7669-9w88x"]
Sep 08 15:52:26.779263 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:52:26.779228    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9424134d_5bcd_44ff_bbd8_0e6c7ef9cd63.slice/crio-9f4004915be2d8b877500c7e7ae8bd6af8cf0f21a4aedb154d6dc5ebe6926464 WatchSource:0}: Error finding container 9f4004915be2d8b877500c7e7ae8bd6af8cf0f21a4aedb154d6dc5ebe6926464: Status 404 returned error can't find the container with id 9f4004915be2d8b877500c7e7ae8bd6af8cf0f21a4aedb154d6dc5ebe6926464
Sep 08 15:52:27.080816 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:27.080735    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75db7f7669-9w88x" event={"ID":"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63","Type":"ContainerStarted","Data":"9f4004915be2d8b877500c7e7ae8bd6af8cf0f21a4aedb154d6dc5ebe6926464"}
Sep 08 15:52:30.099069 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:30.099018    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75db7f7669-9w88x" event={"ID":"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63","Type":"ContainerStarted","Data":"f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d"}
Sep 08 15:52:30.099462 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:30.099131    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:52:30.118501 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:30.118448    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75db7f7669-9w88x" podStartSLOduration=1.543028871 podStartE2EDuration="4.118434438s" totalImagesPullingTime="2.575405567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:52:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:52:30.117947561 +0000 UTC m=+459.614355529" watchObservedRunningTime="2026-09-08 15:52:30.118434438 +0000 UTC m=+459.614842404"
Sep 08 15:52:41.105572 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:52:41.105542    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 15:53:02.093411 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.093379    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9g54m"]
Sep 08 15:53:02.124205 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.124173    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9g54m"]
Sep 08 15:53:02.124367 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.124291    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.126403 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.126377    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 08 15:53:02.126540 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.126435    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6dgjp\""
Sep 08 15:53:02.247602 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.247566    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b382fb-6099-4697-91f1-c4d30851a597-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9g54m\" (UID: \"26b382fb-6099-4697-91f1-c4d30851a597\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.247787 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.247621    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dj49\" (UniqueName: \"kubernetes.io/projected/26b382fb-6099-4697-91f1-c4d30851a597-kube-api-access-8dj49\") pod \"payload-processing-5b54b7fbb7-9g54m\" (UID: \"26b382fb-6099-4697-91f1-c4d30851a597\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.348828 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.348750    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b382fb-6099-4697-91f1-c4d30851a597-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9g54m\" (UID: \"26b382fb-6099-4697-91f1-c4d30851a597\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.348828 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.348795    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dj49\" (UniqueName: \"kubernetes.io/projected/26b382fb-6099-4697-91f1-c4d30851a597-kube-api-access-8dj49\") pod \"payload-processing-5b54b7fbb7-9g54m\" (UID: \"26b382fb-6099-4697-91f1-c4d30851a597\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.349510 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.349489    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26b382fb-6099-4697-91f1-c4d30851a597-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-9g54m\" (UID: \"26b382fb-6099-4697-91f1-c4d30851a597\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.356551 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.356528    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dj49\" (UniqueName: \"kubernetes.io/projected/26b382fb-6099-4697-91f1-c4d30851a597-kube-api-access-8dj49\") pod \"payload-processing-5b54b7fbb7-9g54m\" (UID: \"26b382fb-6099-4697-91f1-c4d30851a597\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.435551 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.435518    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:02.707611 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:02.707572    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-9g54m"]
Sep 08 15:53:03.250686 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:03.250652    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m" event={"ID":"26b382fb-6099-4697-91f1-c4d30851a597","Type":"ContainerStarted","Data":"794d603a817ff28e62d88ff6b5fa8ea805ab8473c5b343967ba75e3536f23be3"}
Sep 08 15:53:06.268103 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:06.268064    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m" event={"ID":"26b382fb-6099-4697-91f1-c4d30851a597","Type":"ContainerStarted","Data":"de5a06ce7ad10bd09667102cec6ea8f59c1454e86ddd72ee85a7f9b3028d7e04"}
Sep 08 15:53:06.268528 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:06.268155    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:06.284106 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:06.284054    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m" podStartSLOduration=1.736871117 podStartE2EDuration="4.284037792s" totalImagesPullingTime="2.547166675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:53:06.282538554 +0000 UTC m=+495.778946522" watchObservedRunningTime="2026-09-08 15:53:06.284037792 +0000 UTC m=+495.780445750"
Sep 08 15:53:11.111631 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.111525    2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ksf69 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": read tcp 10.134.0.2:49726->10.134.0.25:15021: read: connection reset by peer" start-of-body=
Sep 08 15:53:11.111631 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.111599    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" podUID="e8c13238-ca5b-4de2-a21f-4f187ca413b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": read tcp 10.134.0.2:49726->10.134.0.25:15021: read: connection reset by peer"
Sep 08 15:53:11.294653 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.294570    2552 generic.go:356] "Generic (PLEG): container finished" podID="e8c13238-ca5b-4de2-a21f-4f187ca413b6" containerID="2df33cc801b255fd77c9119b31c6d774f94a833abdd578da57a9783d30098c62" exitCode=0
Sep 08 15:53:11.294653 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.294642    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" event={"ID":"e8c13238-ca5b-4de2-a21f-4f187ca413b6","Type":"ContainerDied","Data":"2df33cc801b255fd77c9119b31c6d774f94a833abdd578da57a9783d30098c62"}
Sep 08 15:53:11.295039 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.295013    2552 scope.go:117] "RemoveContainer" containerID="2df33cc801b255fd77c9119b31c6d774f94a833abdd578da57a9783d30098c62"
Sep 08 15:53:11.296037 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.296000    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:53:11.296102 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.296080    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:53:11.296138 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:11.296113    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:53:12.300776 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:12.300740    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" event={"ID":"e8c13238-ca5b-4de2-a21f-4f187ca413b6","Type":"ContainerStarted","Data":"ea27f223796c9eb570663bd1c9325d2347b7011e5dfb9e2e3ec9470f91ab3f47"}
Sep 08 15:53:12.972105 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:12.972065    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:53:12.972333 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:12.972313    2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ksf69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 08 15:53:12.972386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:12.972355    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" podUID="e8c13238-ca5b-4de2-a21f-4f187ca413b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 08 15:53:13.971747 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:13.971713    2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ksf69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 08 15:53:13.972210 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:13.971766    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" podUID="e8c13238-ca5b-4de2-a21f-4f187ca413b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 08 15:53:14.972047 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:14.971988    2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ksf69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 08 15:53:14.972429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:14.972070    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" podUID="e8c13238-ca5b-4de2-a21f-4f187ca413b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 08 15:53:15.971562 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:15.971523    2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ksf69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 08 15:53:15.971736 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:15.971578    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" podUID="e8c13238-ca5b-4de2-a21f-4f187ca413b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 08 15:53:16.971796 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:16.971758    2552 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ksf69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 08 15:53:16.972186 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:16.971816    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69" podUID="e8c13238-ca5b-4de2-a21f-4f187ca413b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 08 15:53:17.274781 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:17.274690    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-9g54m"
Sep 08 15:53:17.974652 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:17.974615    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:53:17.975079 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:17.974909    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:53:17.975529 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:17.975507    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ksf69"
Sep 08 15:53:24.563442 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.563411    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"]
Sep 08 15:53:24.566261 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.566238    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.569146 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.569124    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 15:53:24.569365 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.569129    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 08 15:53:24.569365 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.569140    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-668bh\""
Sep 08 15:53:24.569541 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.569156    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 15:53:24.577175 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.577154    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"]
Sep 08 15:53:24.648160 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.648128    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.648417 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.648398    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.648568 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.648554    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.648680 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.648665    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6g79\" (UniqueName: \"kubernetes.io/projected/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-kube-api-access-v6g79\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.648814 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.648798    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.648936 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.648901    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.750449 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.750417    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.750449 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.750449    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.750646 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.750506    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.750646 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.750612    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.750743 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.750668    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.750743 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.750689    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6g79\" (UniqueName: \"kubernetes.io/projected/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-kube-api-access-v6g79\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.750991 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.750972    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.751099 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.751002    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.751099 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.751069    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.752836 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.752818    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.753130 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.753111    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.757898 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.757875    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6g79\" (UniqueName: \"kubernetes.io/projected/b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d-kube-api-access-v6g79\") pod \"e2e-distinct-simulated-kserve-59975d69f9-cdrxh\" (UID: \"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:24.879112 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:24.879076    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:25.148332 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:25.148280    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"]
Sep 08 15:53:25.150497 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:53:25.150469    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a009d8_3957_48e6_ae8e_5a2a3c92ef6d.slice/crio-d1b421545b0c36f3e1ac0f96fab18454bce38a5acec85ba9bb3f990dcd89feeb WatchSource:0}: Error finding container d1b421545b0c36f3e1ac0f96fab18454bce38a5acec85ba9bb3f990dcd89feeb: Status 404 returned error can't find the container with id d1b421545b0c36f3e1ac0f96fab18454bce38a5acec85ba9bb3f990dcd89feeb
Sep 08 15:53:25.364327 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:25.364286    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh" event={"ID":"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d","Type":"ContainerStarted","Data":"d1b421545b0c36f3e1ac0f96fab18454bce38a5acec85ba9bb3f990dcd89feeb"}
Sep 08 15:53:26.371017 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:26.370985    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh" event={"ID":"b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d","Type":"ContainerStarted","Data":"eb138e158fa2e750c889ba415b462c364b01807bbd649269c14dfe4d4b48e80e"}
Sep 08 15:53:26.389915 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:26.389865    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh" podStartSLOduration=1.392036404 podStartE2EDuration="2.389848315s" totalImagesPullingTime="997.811911ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:53:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:53:26.388559215 +0000 UTC m=+515.884967184" watchObservedRunningTime="2026-09-08 15:53:26.389848315 +0000 UTC m=+515.886256281"
Sep 08 15:53:34.879843 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:34.879796    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:34.880445 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:34.879915    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:34.892476 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:34.892446    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:53:35.424851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:53:35.424822    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-cdrxh"
Sep 08 15:54:00.270141 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.270105    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"]
Sep 08 15:54:00.272816 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.272799    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.284801 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.284756    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 08 15:54:00.288429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.288402    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"]
Sep 08 15:54:00.384182 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.384147    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.384182 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.384187    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.384422 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.384230    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.384422 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.384272    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-852st\" (UniqueName: \"kubernetes.io/projected/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-kube-api-access-852st\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.384422 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.384298    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.384422 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.384327    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485191 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485156    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485191 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485195    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-852st\" (UniqueName: \"kubernetes.io/projected/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-kube-api-access-852st\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485400 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485233    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485400 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485258    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485400 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485342    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485400 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485369    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485637 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485615    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485701 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485679    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.485739 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.485693    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.487622 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.487598    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.487908 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.487888    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.493514 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.493486    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-852st\" (UniqueName: \"kubernetes.io/projected/f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d-kube-api-access-852st\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f\" (UID: \"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.587152 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.587071    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:00.859985 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:00.859960    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"]
Sep 08 15:54:00.862399 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:54:00.862369    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6dc8cd4_9eb8_445d_b3b6_5dbf80a2e44d.slice/crio-ba85d079c1d1c122a9eb0ebba4cada4e70d520b35c863c5baab6dda3fb950aa7 WatchSource:0}: Error finding container ba85d079c1d1c122a9eb0ebba4cada4e70d520b35c863c5baab6dda3fb950aa7: Status 404 returned error can't find the container with id ba85d079c1d1c122a9eb0ebba4cada4e70d520b35c863c5baab6dda3fb950aa7
Sep 08 15:54:01.536118 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:01.536084    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f" event={"ID":"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d","Type":"ContainerStarted","Data":"352ad366bc6bf3fc0a98a9b4065e929387b2eb75429972fbcc38d8859d3d16cc"}
Sep 08 15:54:01.536505 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:01.536140    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f" event={"ID":"f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d","Type":"ContainerStarted","Data":"ba85d079c1d1c122a9eb0ebba4cada4e70d520b35c863c5baab6dda3fb950aa7"}
Sep 08 15:54:01.552598 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:01.552546    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f" podStartSLOduration=1.102649218 podStartE2EDuration="1.552530465s" totalImagesPullingTime="449.881247ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:54:01.551579714 +0000 UTC m=+551.047987682" watchObservedRunningTime="2026-09-08 15:54:01.552530465 +0000 UTC m=+551.048938432"
Sep 08 15:54:10.588092 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:10.588060    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:10.588577 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:10.588141    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:10.600502 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:10.600475    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:11.596212 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:11.596181    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f"
Sep 08 15:54:18.866617 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.866583    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"]
Sep 08 15:54:18.870244 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.870219    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:18.872418 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.872395    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 08 15:54:18.878288 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.878240    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"]
Sep 08 15:54:18.962556 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.962519    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrxg5\" (UniqueName: \"kubernetes.io/projected/20f81e62-734b-4fe0-8903-fd0775d5e750-kube-api-access-qrxg5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:18.962556 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.962557    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f81e62-734b-4fe0-8903-fd0775d5e750-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:18.962763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.962587    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:18.962763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.962604    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:18.962763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.962640    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:18.962763 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:18.962739    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.063712 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.063675    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.063890 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.063765    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.063890 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.063818    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrxg5\" (UniqueName: \"kubernetes.io/projected/20f81e62-734b-4fe0-8903-fd0775d5e750-kube-api-access-qrxg5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.064070 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.064050    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f81e62-734b-4fe0-8903-fd0775d5e750-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.064146 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.064110    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.064146 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.064135    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.064146 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.064137    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.064289 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.064167    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.064493 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.064472    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.066257 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.066236    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f81e62-734b-4fe0-8903-fd0775d5e750-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.066431 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.066414    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f81e62-734b-4fe0-8903-fd0775d5e750-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.071834 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.071809    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrxg5\" (UniqueName: \"kubernetes.io/projected/20f81e62-734b-4fe0-8903-fd0775d5e750-kube-api-access-qrxg5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk\" (UID: \"20f81e62-734b-4fe0-8903-fd0775d5e750\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.181374 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.181283    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:19.451861 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.451836    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"]
Sep 08 15:54:19.453715 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:54:19.453684    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20f81e62_734b_4fe0_8903_fd0775d5e750.slice/crio-411e9f6f75fa2fb06dbbc1eb3969c8893bc86da64a171282553a668ad284b368 WatchSource:0}: Error finding container 411e9f6f75fa2fb06dbbc1eb3969c8893bc86da64a171282553a668ad284b368: Status 404 returned error can't find the container with id 411e9f6f75fa2fb06dbbc1eb3969c8893bc86da64a171282553a668ad284b368
Sep 08 15:54:19.623008 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:19.622968    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk" event={"ID":"20f81e62-734b-4fe0-8903-fd0775d5e750","Type":"ContainerStarted","Data":"411e9f6f75fa2fb06dbbc1eb3969c8893bc86da64a171282553a668ad284b368"}
Sep 08 15:54:20.629302 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:20.629268    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk" event={"ID":"20f81e62-734b-4fe0-8903-fd0775d5e750","Type":"ContainerStarted","Data":"d6709f5bb48b54ae2ea0be76f479c372f109f40e376112d994ed58af3440f45e"}
Sep 08 15:54:20.649476 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:20.649426    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk" podStartSLOduration=2.458788813 podStartE2EDuration="2.649408749s" totalImagesPullingTime="190.619936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:54:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:54:20.647158289 +0000 UTC m=+570.143566255" watchObservedRunningTime="2026-09-08 15:54:20.649408749 +0000 UTC m=+570.145816730"
Sep 08 15:54:29.182070 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:29.182009    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:29.182070 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:29.182074    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:29.194810 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:29.194783    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:54:29.683233 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:54:29.683204    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk"
Sep 08 15:55:16.753263 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.753182    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"]
Sep 08 15:55:16.756592 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.756572    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"
Sep 08 15:55:16.758751 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.758727    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ftfb4\""
Sep 08 15:55:16.758846 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.758755    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 08 15:55:16.766597 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.766575    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"]
Sep 08 15:55:16.795829 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.795799    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cjw6\" (UniqueName: \"kubernetes.io/projected/ada175f5-1375-48d0-93ee-9ffc7e8c2719-kube-api-access-4cjw6\") pod \"dns-operator-controller-manager-7b84cd7c9c-ftwxm\" (UID: \"ada175f5-1375-48d0-93ee-9ffc7e8c2719\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"
Sep 08 15:55:16.896857 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.896825    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cjw6\" (UniqueName: \"kubernetes.io/projected/ada175f5-1375-48d0-93ee-9ffc7e8c2719-kube-api-access-4cjw6\") pod \"dns-operator-controller-manager-7b84cd7c9c-ftwxm\" (UID: \"ada175f5-1375-48d0-93ee-9ffc7e8c2719\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"
Sep 08 15:55:16.904773 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:16.904747    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cjw6\" (UniqueName: \"kubernetes.io/projected/ada175f5-1375-48d0-93ee-9ffc7e8c2719-kube-api-access-4cjw6\") pod \"dns-operator-controller-manager-7b84cd7c9c-ftwxm\" (UID: \"ada175f5-1375-48d0-93ee-9ffc7e8c2719\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"
Sep 08 15:55:17.066942 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:17.066851    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"
Sep 08 15:55:17.343794 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:17.343770    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"]
Sep 08 15:55:17.346666 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:55:17.346633    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada175f5_1375_48d0_93ee_9ffc7e8c2719.slice/crio-849a408a352397462a89890fdc96276dbb58a3fbe5128ac37c95234d2f824ccb WatchSource:0}: Error finding container 849a408a352397462a89890fdc96276dbb58a3fbe5128ac37c95234d2f824ccb: Status 404 returned error can't find the container with id 849a408a352397462a89890fdc96276dbb58a3fbe5128ac37c95234d2f824ccb
Sep 08 15:55:17.896631 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:17.896594    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm" event={"ID":"ada175f5-1375-48d0-93ee-9ffc7e8c2719","Type":"ContainerStarted","Data":"849a408a352397462a89890fdc96276dbb58a3fbe5128ac37c95234d2f824ccb"}
Sep 08 15:55:18.902538 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:18.902505    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm" event={"ID":"ada175f5-1375-48d0-93ee-9ffc7e8c2719","Type":"ContainerStarted","Data":"290094e7bf38b414486de6d952c42c34bcebb98a91bd4da31a5778b10ce0520a"}
Sep 08 15:55:18.902874 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:18.902639    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"
Sep 08 15:55:18.930902 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:18.930845    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm" podStartSLOduration=1.4491422809999999 podStartE2EDuration="2.930824974s" totalImagesPullingTime="1.481682693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:55:18.927456382 +0000 UTC m=+628.423864351" watchObservedRunningTime="2026-09-08 15:55:18.930824974 +0000 UTC m=+628.427232941"
Sep 08 15:55:24.756981 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.756941    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"]
Sep 08 15:55:24.760679 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.760660    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:24.763895 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.763867    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t8mht\""
Sep 08 15:55:24.782833 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.782804    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"]
Sep 08 15:55:24.872730 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.872700    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8khvc\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:24.872867 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.872749    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvm6x\" (UniqueName: \"kubernetes.io/projected/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-kube-api-access-jvm6x\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8khvc\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:24.973608 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.973578    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvm6x\" (UniqueName: \"kubernetes.io/projected/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-kube-api-access-jvm6x\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8khvc\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:24.973762 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.973673    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8khvc\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:24.974043 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.974007    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8khvc\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:24.987149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:24.987117    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvm6x\" (UniqueName: \"kubernetes.io/projected/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-kube-api-access-jvm6x\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8khvc\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:25.071021 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:25.070950    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:25.373568 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:25.373522    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"]
Sep 08 15:55:25.375927 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:55:25.375904    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4d6656_58a3_4327_ae63_fa4a0165f5b8.slice/crio-97141ce5f6993b195194310925dc1fc5e6a220421e350d2e2656054e968730d1 WatchSource:0}: Error finding container 97141ce5f6993b195194310925dc1fc5e6a220421e350d2e2656054e968730d1: Status 404 returned error can't find the container with id 97141ce5f6993b195194310925dc1fc5e6a220421e350d2e2656054e968730d1
Sep 08 15:55:25.939897 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:25.939846    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc" event={"ID":"2e4d6656-58a3-4327-ae63-fa4a0165f5b8","Type":"ContainerStarted","Data":"97141ce5f6993b195194310925dc1fc5e6a220421e350d2e2656054e968730d1"}
Sep 08 15:55:29.909637 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:29.909602    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ftwxm"
Sep 08 15:55:29.966476 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:29.966438    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc" event={"ID":"2e4d6656-58a3-4327-ae63-fa4a0165f5b8","Type":"ContainerStarted","Data":"b3bd9a4a07a94a35950996f658b453fbb6a1762cc18ffdef5de66dbfed5e2734"}
Sep 08 15:55:29.966667 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:29.966593    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:55:29.998466 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:29.998404    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc" podStartSLOduration=2.420348259 podStartE2EDuration="5.998382752s" totalImagesPullingTime="3.578034493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:55:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:55:29.997513683 +0000 UTC m=+639.493921685" watchObservedRunningTime="2026-09-08 15:55:29.998382752 +0000 UTC m=+639.494790720"
Sep 08 15:55:40.974486 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:55:40.974454    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 15:56:05.178209 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.178177    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"]
Sep 08 15:56:05.181906 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.181879    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.191714 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.191686    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"]
Sep 08 15:56:05.251094 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.251055    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cb4s\" (UniqueName: \"kubernetes.io/projected/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-kube-api-access-9cb4s\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.251259 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.251120    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.251259 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.251143    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.352308 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.352274    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.352308 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.352314    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.352549 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.352455    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cb4s\" (UniqueName: \"kubernetes.io/projected/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-kube-api-access-9cb4s\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.353082 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.353052    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.355017 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.354988    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.365011 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.364987    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cb4s\" (UniqueName: \"kubernetes.io/projected/1eb678b0-f5b9-437a-8388-fb6b5b6d20e2-kube-api-access-9cb4s\") pod \"kuadrant-console-plugin-fbdd4f445-cdkr6\" (UID: \"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.493082 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.492977    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"
Sep 08 15:56:05.763452 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.763427    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6"]
Sep 08 15:56:05.765538 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:56:05.765509    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1eb678b0_f5b9_437a_8388_fb6b5b6d20e2.slice/crio-524188d6e00c1e778f343472f51e7bad7a608f4f7f9b1ebf2d21ed28cea4110b WatchSource:0}: Error finding container 524188d6e00c1e778f343472f51e7bad7a608f4f7f9b1ebf2d21ed28cea4110b: Status 404 returned error can't find the container with id 524188d6e00c1e778f343472f51e7bad7a608f4f7f9b1ebf2d21ed28cea4110b
Sep 08 15:56:05.766879 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:05.766863    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:56:06.151259 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:06.151224    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6" event={"ID":"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2","Type":"ContainerStarted","Data":"524188d6e00c1e778f343472f51e7bad7a608f4f7f9b1ebf2d21ed28cea4110b"}
Sep 08 15:56:12.189748 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:12.189714    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6" event={"ID":"1eb678b0-f5b9-437a-8388-fb6b5b6d20e2","Type":"ContainerStarted","Data":"113a2182178ee9078e60aa1b369e06cd1fb5ea4b1b617ce36613345ad0d29236"}
Sep 08 15:56:12.205667 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:12.205619    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-cdkr6" podStartSLOduration=1.394780236 podStartE2EDuration="7.205603646s" totalImagesPullingTime="5.81082341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:56:12.204082592 +0000 UTC m=+681.700490563" watchObservedRunningTime="2026-09-08 15:56:12.205603646 +0000 UTC m=+681.702011613"
Sep 08 15:56:12.236571 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:12.236532    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"]
Sep 08 15:56:12.236986 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:12.236952    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d" podUID="a91bab1c-3256-476e-875c-67aa5d835154" containerName="kuadrant-console-plugin" containerID="cri-o://a11c3c33acb4ae696c7b67db835e26bc4c61971e501d51ae8675260d74647902" gracePeriod=30
Sep 08 15:56:25.201281 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.201252    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"]
Sep 08 15:56:25.204775 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.204755    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"
Sep 08 15:56:25.206889 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.206862    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 15:56:25.207008 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.206888    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:56:25.216480 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.216460    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"]
Sep 08 15:56:25.273122 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.273093    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"]
Sep 08 15:56:25.276380 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.276364    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.278869 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.278845    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c96e2b\""
Sep 08 15:56:25.279151 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.279135    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c96e2b-dockercfg-mqz9d\""
Sep 08 15:56:25.307714 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.307681    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"]
Sep 08 15:56:25.311127 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.311107    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.327648 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.327627    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"]
Sep 08 15:56:25.347065 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.347040    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"]
Sep 08 15:56:25.359454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.359427    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2b2d5d-6242-46bc-afdc-7fa0b1eac243-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc\" (UID: \"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"
Sep 08 15:56:25.359576 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.359482    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.359576 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.359560    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkzns\" (UniqueName: \"kubernetes.io/projected/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-kube-api-access-wkzns\") pod \"payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.460622 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.460533    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2b2d5d-6242-46bc-afdc-7fa0b1eac243-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc\" (UID: \"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"
Sep 08 15:56:25.460622 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.460614    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.460843 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.460664    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.460843 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.460725    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkzns\" (UniqueName: \"kubernetes.io/projected/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-kube-api-access-wkzns\") pod \"payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.460843 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.460754    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl8nc\" (UniqueName: \"kubernetes.io/projected/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-kube-api-access-sl8nc\") pod \"payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.461386 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.461358    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2b2d5d-6242-46bc-afdc-7fa0b1eac243-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc\" (UID: \"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"
Sep 08 15:56:25.461498 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.461461    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.469137 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.469115    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkzns\" (UniqueName: \"kubernetes.io/projected/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-kube-api-access-wkzns\") pod \"payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.515463 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.515428    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"
Sep 08 15:56:25.561734 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.561702    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.561886 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.561761    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl8nc\" (UniqueName: \"kubernetes.io/projected/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-kube-api-access-sl8nc\") pod \"payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.562431 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.562410    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.577696 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.574618    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl8nc\" (UniqueName: \"kubernetes.io/projected/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-kube-api-access-sl8nc\") pod \"payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.587632 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.587559    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:25.620857 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.620825    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:25.793179 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.793142    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"]
Sep 08 15:56:25.795596 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:56:25.795567    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2b2d5d_6242_46bc_afdc_7fa0b1eac243.slice/crio-148e76861d79adc35f6d8cee2947bd224515bde0fbf4e3cb00ff8af1745d6b64 WatchSource:0}: Error finding container 148e76861d79adc35f6d8cee2947bd224515bde0fbf4e3cb00ff8af1745d6b64: Status 404 returned error can't find the container with id 148e76861d79adc35f6d8cee2947bd224515bde0fbf4e3cb00ff8af1745d6b64
Sep 08 15:56:25.871501 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.871472    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"]
Sep 08 15:56:25.874606 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:56:25.874579    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00cea2d1_13f2_4b08_a0fc_4eefcbd3f192.slice/crio-e492b1fcfcd1baf134f8ccad814008f39d8a84f89090866d3a845fa7a6f47f41 WatchSource:0}: Error finding container e492b1fcfcd1baf134f8ccad814008f39d8a84f89090866d3a845fa7a6f47f41: Status 404 returned error can't find the container with id e492b1fcfcd1baf134f8ccad814008f39d8a84f89090866d3a845fa7a6f47f41
Sep 08 15:56:25.914783 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:25.914759    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"]
Sep 08 15:56:25.916714 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:56:25.916689    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84cdf92a_a938_4b95_b6b2_b45fa3c7659c.slice/crio-5f33998e3774c5532086b9c30b137e29906a54e14ce5cd326e322dad2cae8105 WatchSource:0}: Error finding container 5f33998e3774c5532086b9c30b137e29906a54e14ce5cd326e322dad2cae8105: Status 404 returned error can't find the container with id 5f33998e3774c5532086b9c30b137e29906a54e14ce5cd326e322dad2cae8105
Sep 08 15:56:26.263636 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.263594    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc" event={"ID":"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243","Type":"ContainerStarted","Data":"148e76861d79adc35f6d8cee2947bd224515bde0fbf4e3cb00ff8af1745d6b64"}
Sep 08 15:56:26.265649 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.265617    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g" event={"ID":"84cdf92a-a938-4b95-b6b2-b45fa3c7659c","Type":"ContainerStarted","Data":"71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d"}
Sep 08 15:56:26.265800 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.265675    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g" event={"ID":"84cdf92a-a938-4b95-b6b2-b45fa3c7659c","Type":"ContainerStarted","Data":"5f33998e3774c5532086b9c30b137e29906a54e14ce5cd326e322dad2cae8105"}
Sep 08 15:56:26.265800 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.265737    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:26.267368 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.267345    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd" event={"ID":"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192","Type":"ContainerStarted","Data":"ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99"}
Sep 08 15:56:26.267510 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.267379    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd" event={"ID":"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192","Type":"ContainerStarted","Data":"e492b1fcfcd1baf134f8ccad814008f39d8a84f89090866d3a845fa7a6f47f41"}
Sep 08 15:56:26.267510 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.267491    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:26.283648 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.283597    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g" podStartSLOduration=1.2835803129999999 podStartE2EDuration="1.283580313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:56:26.282780219 +0000 UTC m=+695.779188188" watchObservedRunningTime="2026-09-08 15:56:26.283580313 +0000 UTC m=+695.779988286"
Sep 08 15:56:26.299115 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:26.299067    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd" podStartSLOduration=1.2990505589999999 podStartE2EDuration="1.299050559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:56:26.297308103 +0000 UTC m=+695.793716072" watchObservedRunningTime="2026-09-08 15:56:26.299050559 +0000 UTC m=+695.795458598"
Sep 08 15:56:28.355437 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.355403    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"]
Sep 08 15:56:28.359471 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.359451    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.361729 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.361704    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-e6f19b8d-kserve-self-signed-certs\""
Sep 08 15:56:28.366483 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.366461    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"]
Sep 08 15:56:28.492127 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.492091    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tls-certs\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.492307 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.492136    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-home\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.492307 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.492233    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tmp-dir\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.492307 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.492261    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcb9r\" (UniqueName: \"kubernetes.io/projected/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-kube-api-access-dcb9r\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.492432 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.492315    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-model-cache\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.492432 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.492355    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-dshm\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.593892 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.593795    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tls-certs\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.593892 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.593836    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-home\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.593892 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.593873    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tmp-dir\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.594189 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.593988    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcb9r\" (UniqueName: \"kubernetes.io/projected/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-kube-api-access-dcb9r\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.594189 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.594074    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-model-cache\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.594189 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.594128    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-dshm\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.594345 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.594252    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tmp-dir\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.594345 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.594319    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-home\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.594457 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.594407    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-model-cache\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.596237 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.596211    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-dshm\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.596417 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.596396    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tls-certs\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.606204 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.606183    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcb9r\" (UniqueName: \"kubernetes.io/projected/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-kube-api-access-dcb9r\") pod \"e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") " pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.672454 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.672420    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:28.950947 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:28.950920    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"]
Sep 08 15:56:28.953232 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:56:28.953208    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5868dad_11fa_44b5_b4e8_7ae6a30bea87.slice/crio-329667adefdacef1f6d1794f28d196924ac373039a3acf106a7a940f37da5a98 WatchSource:0}: Error finding container 329667adefdacef1f6d1794f28d196924ac373039a3acf106a7a940f37da5a98: Status 404 returned error can't find the container with id 329667adefdacef1f6d1794f28d196924ac373039a3acf106a7a940f37da5a98
Sep 08 15:56:29.286174 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:29.286085    2552 generic.go:356] "Generic (PLEG): container finished" podID="2f2b2d5d-6242-46bc-afdc-7fa0b1eac243" containerID="1be28e63fa0cc90524ecfe76196bc171b03392023cadde2f7c2ccbc1cd5c2caa" exitCode=0
Sep 08 15:56:29.286330 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:29.286172    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc" event={"ID":"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243","Type":"ContainerDied","Data":"1be28e63fa0cc90524ecfe76196bc171b03392023cadde2f7c2ccbc1cd5c2caa"}
Sep 08 15:56:29.287731 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:29.287706    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr" event={"ID":"b5868dad-11fa-44b5-b4e8-7ae6a30bea87","Type":"ContainerStarted","Data":"65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75"}
Sep 08 15:56:29.287848 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:29.287742    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr" event={"ID":"b5868dad-11fa-44b5-b4e8-7ae6a30bea87","Type":"ContainerStarted","Data":"329667adefdacef1f6d1794f28d196924ac373039a3acf106a7a940f37da5a98"}
Sep 08 15:56:29.316491 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:29.316447    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr" podStartSLOduration=1.316433795 podStartE2EDuration="1.316433795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:56:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:56:29.315725061 +0000 UTC m=+698.812133055" watchObservedRunningTime="2026-09-08 15:56:29.316433795 +0000 UTC m=+698.812841762"
Sep 08 15:56:30.428038 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:30.428008    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"
Sep 08 15:56:30.614392 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:30.614366    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2b2d5d-6242-46bc-afdc-7fa0b1eac243-maas-api-service-ca\") pod \"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243\" (UID: \"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243\") "
Sep 08 15:56:30.614732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:30.614705    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f2b2d5d-6242-46bc-afdc-7fa0b1eac243-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f2b2d5d-6242-46bc-afdc-7fa0b1eac243" (UID: "2f2b2d5d-6242-46bc-afdc-7fa0b1eac243"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:56:30.715343 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:30.715315    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f2b2d5d-6242-46bc-afdc-7fa0b1eac243-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:56:31.298874 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:31.298845    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"
Sep 08 15:56:31.299054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:31.298834    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc" event={"ID":"2f2b2d5d-6242-46bc-afdc-7fa0b1eac243","Type":"ContainerDied","Data":"148e76861d79adc35f6d8cee2947bd224515bde0fbf4e3cb00ff8af1745d6b64"}
Sep 08 15:56:31.299054 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:31.298964    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="148e76861d79adc35f6d8cee2947bd224515bde0fbf4e3cb00ff8af1745d6b64"
Sep 08 15:56:37.275364 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:37.275328    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 15:56:37.276218 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:37.275428    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 15:56:38.673384 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:38.673342    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:38.673781 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:38.673398    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:38.692236 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:38.692212    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:39.358514 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:39.358479    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:56:42.028153 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.028115    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"]
Sep 08 15:56:42.030640 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.030608    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-cb6c353e-hc9jc"]
Sep 08 15:56:42.364263 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.364228    2552 generic.go:356] "Generic (PLEG): container finished" podID="a91bab1c-3256-476e-875c-67aa5d835154" containerID="a11c3c33acb4ae696c7b67db835e26bc4c61971e501d51ae8675260d74647902" exitCode=137
Sep 08 15:56:42.364431 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.364274    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d" event={"ID":"a91bab1c-3256-476e-875c-67aa5d835154","Type":"ContainerDied","Data":"a11c3c33acb4ae696c7b67db835e26bc4c61971e501d51ae8675260d74647902"}
Sep 08 15:56:42.599877 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.599846    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:56:42.625440 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.625411    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert\") pod \"a91bab1c-3256-476e-875c-67aa5d835154\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") "
Sep 08 15:56:42.625593 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.625453    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a91bab1c-3256-476e-875c-67aa5d835154-nginx-conf\") pod \"a91bab1c-3256-476e-875c-67aa5d835154\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") "
Sep 08 15:56:42.625593 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.625525    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m49d\" (UniqueName: \"kubernetes.io/projected/a91bab1c-3256-476e-875c-67aa5d835154-kube-api-access-6m49d\") pod \"a91bab1c-3256-476e-875c-67aa5d835154\" (UID: \"a91bab1c-3256-476e-875c-67aa5d835154\") "
Sep 08 15:56:42.627863 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.627832    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert" (OuterVolumeSpecName: "plugin-serving-cert") pod "a91bab1c-3256-476e-875c-67aa5d835154" (UID: "a91bab1c-3256-476e-875c-67aa5d835154"). InnerVolumeSpecName "plugin-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:56:42.628223 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.628202    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91bab1c-3256-476e-875c-67aa5d835154-kube-api-access-6m49d" (OuterVolumeSpecName: "kube-api-access-6m49d") pod "a91bab1c-3256-476e-875c-67aa5d835154" (UID: "a91bab1c-3256-476e-875c-67aa5d835154"). InnerVolumeSpecName "kube-api-access-6m49d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:56:42.632967 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.632939    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91bab1c-3256-476e-875c-67aa5d835154-nginx-conf" (OuterVolumeSpecName: "nginx-conf") pod "a91bab1c-3256-476e-875c-67aa5d835154" (UID: "a91bab1c-3256-476e-875c-67aa5d835154"). InnerVolumeSpecName "nginx-conf". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:56:42.726359 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.726326    2552 reconciler_common.go:299] "Volume detached for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a91bab1c-3256-476e-875c-67aa5d835154-plugin-serving-cert\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:56:42.726359 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.726357    2552 reconciler_common.go:299] "Volume detached for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a91bab1c-3256-476e-875c-67aa5d835154-nginx-conf\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:56:42.726359 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:42.726368    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m49d\" (UniqueName: \"kubernetes.io/projected/a91bab1c-3256-476e-875c-67aa5d835154-kube-api-access-6m49d\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:56:43.101984 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:43.101943    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f2b2d5d-6242-46bc-afdc-7fa0b1eac243" path="/var/lib/kubelet/pods/2f2b2d5d-6242-46bc-afdc-7fa0b1eac243/volumes"
Sep 08 15:56:43.370963 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:43.370935    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"
Sep 08 15:56:43.371159 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:43.370932    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d" event={"ID":"a91bab1c-3256-476e-875c-67aa5d835154","Type":"ContainerDied","Data":"f26d7e16c6df744ebf5af98c631ce9d787ec5e5dd6610d7687ab1c0da76aaeba"}
Sep 08 15:56:43.371159 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:43.371085    2552 scope.go:117] "RemoveContainer" containerID="a11c3c33acb4ae696c7b67db835e26bc4c61971e501d51ae8675260d74647902"
Sep 08 15:56:43.401179 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:43.401149    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"]
Sep 08 15:56:43.429676 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:43.429628    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-rbf7d"]
Sep 08 15:56:45.100710 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:45.100674    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a91bab1c-3256-476e-875c-67aa5d835154" path="/var/lib/kubelet/pods/a91bab1c-3256-476e-875c-67aa5d835154/volumes"
Sep 08 15:56:52.372927 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.372888    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"]
Sep 08 15:56:52.373464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.373445    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f2b2d5d-6242-46bc-afdc-7fa0b1eac243" containerName="revoke-keys"
Sep 08 15:56:52.373464 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.373465    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f2b2d5d-6242-46bc-afdc-7fa0b1eac243" containerName="revoke-keys"
Sep 08 15:56:52.373573 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.373482    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91bab1c-3256-476e-875c-67aa5d835154" containerName="kuadrant-console-plugin"
Sep 08 15:56:52.373573 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.373488    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91bab1c-3256-476e-875c-67aa5d835154" containerName="kuadrant-console-plugin"
Sep 08 15:56:52.373573 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.373550    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91bab1c-3256-476e-875c-67aa5d835154" containerName="kuadrant-console-plugin"
Sep 08 15:56:52.373573 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.373560    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f2b2d5d-6242-46bc-afdc-7fa0b1eac243" containerName="revoke-keys"
Sep 08 15:56:52.378021 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.378002    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.380409 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.380388    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-423c90\"/\"openshift-service-ca.crt\""
Sep 08 15:56:52.380409 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.380399    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-423c90\"/\"kube-root-ca.crt\""
Sep 08 15:56:52.380987 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.380968    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-423c90\"/\"default-dockercfg-rggg2\""
Sep 08 15:56:52.381104 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.381022    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-423c90\"/\"autores-model-423c90-kserve-self-signed-certs\""
Sep 08 15:56:52.387682 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.387531    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"]
Sep 08 15:56:52.527271 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.527230    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-home\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.527477 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.527285    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tmp-dir\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.527477 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.527319    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tls-certs\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.527477 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.527380    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws2xz\" (UniqueName: \"kubernetes.io/projected/03d38e2e-236f-4df0-b262-fd23c8dd57d0-kube-api-access-ws2xz\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.527477 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.527458    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-dshm\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.527694 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.527527    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-model-cache\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629197 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629100    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-model-cache\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629197 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629153    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-home\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629204    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tmp-dir\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629232    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tls-certs\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629313    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws2xz\" (UniqueName: \"kubernetes.io/projected/03d38e2e-236f-4df0-b262-fd23c8dd57d0-kube-api-access-ws2xz\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629429 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629351    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-dshm\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629649 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629620    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-model-cache\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629832 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629633    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tmp-dir\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.629832 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.629686    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-home\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.631752 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.631723    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-dshm\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.632104 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.632087    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tls-certs\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.636509 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.636486    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws2xz\" (UniqueName: \"kubernetes.io/projected/03d38e2e-236f-4df0-b262-fd23c8dd57d0-kube-api-access-ws2xz\") pod \"autores-model-423c90-kserve-79fc774f75-dnpsw\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") " pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.690147 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.690109    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:56:52.969144 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:52.969118    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"]
Sep 08 15:56:52.970457 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:56:52.970429    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03d38e2e_236f_4df0_b262_fd23c8dd57d0.slice/crio-24df2f286b534d7885c18ea44c2819ebc4b97ef1fa07ba870b813493d8eb941d WatchSource:0}: Error finding container 24df2f286b534d7885c18ea44c2819ebc4b97ef1fa07ba870b813493d8eb941d: Status 404 returned error can't find the container with id 24df2f286b534d7885c18ea44c2819ebc4b97ef1fa07ba870b813493d8eb941d
Sep 08 15:56:53.424175 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:53.424145    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw" event={"ID":"03d38e2e-236f-4df0-b262-fd23c8dd57d0","Type":"ContainerStarted","Data":"f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7"}
Sep 08 15:56:53.424175 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:53.424183    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw" event={"ID":"03d38e2e-236f-4df0-b262-fd23c8dd57d0","Type":"ContainerStarted","Data":"24df2f286b534d7885c18ea44c2819ebc4b97ef1fa07ba870b813493d8eb941d"}
Sep 08 15:56:53.440411 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:56:53.440358    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw" podStartSLOduration=1.4403413299999999 podStartE2EDuration="1.44034133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:56:53.439313815 +0000 UTC m=+722.935721782" watchObservedRunningTime="2026-09-08 15:56:53.44034133 +0000 UTC m=+722.936749298"
Sep 08 15:57:02.690390 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:02.690347    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:57:02.690878 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:02.690408    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:57:02.703119 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:02.703090    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:57:03.489427 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:03.489394    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:57:10.982974 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:10.982938    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"]
Sep 08 15:57:10.986830 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:10.986809    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"
Sep 08 15:57:10.989093 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:10.989073    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:57:10.989227 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:10.989123    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 15:57:10.995160 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:10.995132    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"]
Sep 08 15:57:11.020043 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:11.019986    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98595413-fd03-45c0-bbe2-ff178f3c1bd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl\" (UID: \"98595413-fd03-45c0-bbe2-ff178f3c1bd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"
Sep 08 15:57:11.120730 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:11.120695    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98595413-fd03-45c0-bbe2-ff178f3c1bd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl\" (UID: \"98595413-fd03-45c0-bbe2-ff178f3c1bd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"
Sep 08 15:57:11.121393 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:11.121372    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98595413-fd03-45c0-bbe2-ff178f3c1bd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl\" (UID: \"98595413-fd03-45c0-bbe2-ff178f3c1bd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"
Sep 08 15:57:11.303977 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:11.303892    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"
Sep 08 15:57:11.575109 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:11.575085    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"]
Sep 08 15:57:11.576934 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:57:11.576908    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98595413_fd03_45c0_bbe2_ff178f3c1bd0.slice/crio-72c22e1ba1ddf151026e2cebb17239bb7d79c45197168c151b3c66b43a412eb8 WatchSource:0}: Error finding container 72c22e1ba1ddf151026e2cebb17239bb7d79c45197168c151b3c66b43a412eb8: Status 404 returned error can't find the container with id 72c22e1ba1ddf151026e2cebb17239bb7d79c45197168c151b3c66b43a412eb8
Sep 08 15:57:12.524705 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:12.524616    2552 generic.go:356] "Generic (PLEG): container finished" podID="98595413-fd03-45c0-bbe2-ff178f3c1bd0" containerID="54defc319c367d43390c0dd098e45c32bbf01965fefdb72be5c291dd4b1e1f68" exitCode=0
Sep 08 15:57:12.525085 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:12.524699    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl" event={"ID":"98595413-fd03-45c0-bbe2-ff178f3c1bd0","Type":"ContainerDied","Data":"54defc319c367d43390c0dd098e45c32bbf01965fefdb72be5c291dd4b1e1f68"}
Sep 08 15:57:12.525085 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:12.524757    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl" event={"ID":"98595413-fd03-45c0-bbe2-ff178f3c1bd0","Type":"ContainerStarted","Data":"72c22e1ba1ddf151026e2cebb17239bb7d79c45197168c151b3c66b43a412eb8"}
Sep 08 15:57:13.684873 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:13.684848    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"
Sep 08 15:57:13.747902 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:13.747868    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98595413-fd03-45c0-bbe2-ff178f3c1bd0-maas-api-service-ca\") pod \"98595413-fd03-45c0-bbe2-ff178f3c1bd0\" (UID: \"98595413-fd03-45c0-bbe2-ff178f3c1bd0\") "
Sep 08 15:57:13.748242 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:13.748218    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98595413-fd03-45c0-bbe2-ff178f3c1bd0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "98595413-fd03-45c0-bbe2-ff178f3c1bd0" (UID: "98595413-fd03-45c0-bbe2-ff178f3c1bd0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:13.849703 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:13.849617    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98595413-fd03-45c0-bbe2-ff178f3c1bd0-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:14.539309 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:14.539275    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl" event={"ID":"98595413-fd03-45c0-bbe2-ff178f3c1bd0","Type":"ContainerDied","Data":"72c22e1ba1ddf151026e2cebb17239bb7d79c45197168c151b3c66b43a412eb8"}
Sep 08 15:57:14.539309 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:14.539297    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"
Sep 08 15:57:14.539578 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:14.539327    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72c22e1ba1ddf151026e2cebb17239bb7d79c45197168c151b3c66b43a412eb8"
Sep 08 15:57:17.908694 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:17.908642    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"]
Sep 08 15:57:17.911113 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:17.911091    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-173f5911-m4fvl"]
Sep 08 15:57:19.101161 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:19.101121    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98595413-fd03-45c0-bbe2-ff178f3c1bd0" path="/var/lib/kubelet/pods/98595413-fd03-45c0-bbe2-ff178f3c1bd0/volumes"
Sep 08 15:57:26.869610 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.869569    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"]
Sep 08 15:57:26.870235 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.870213    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98595413-fd03-45c0-bbe2-ff178f3c1bd0" containerName="revoke-keys"
Sep 08 15:57:26.870292 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.870238    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="98595413-fd03-45c0-bbe2-ff178f3c1bd0" containerName="revoke-keys"
Sep 08 15:57:26.870376 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.870364    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="98595413-fd03-45c0-bbe2-ff178f3c1bd0" containerName="revoke-keys"
Sep 08 15:57:26.874268 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.874243    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"
Sep 08 15:57:26.876983 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.876952    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 15:57:26.877119 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.876982    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:57:26.879168 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.879146    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"]
Sep 08 15:57:26.979136 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:26.979101    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-423c90-lzfdf\" (UID: \"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"
Sep 08 15:57:27.080449 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:27.080410    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-423c90-lzfdf\" (UID: \"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"
Sep 08 15:57:27.081225 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:27.081204    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-423c90-lzfdf\" (UID: \"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"
Sep 08 15:57:27.187324 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:27.187251    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"
Sep 08 15:57:27.459851 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:27.459824    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"]
Sep 08 15:57:27.462686 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:57:27.462656    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a91e52_ae9d_45a8_ba0c_3a6a19f2e540.slice/crio-8915529462f8c37f66480ecdc3bb261fc7f44dca4c5f49791c5d5b17f20b8d73 WatchSource:0}: Error finding container 8915529462f8c37f66480ecdc3bb261fc7f44dca4c5f49791c5d5b17f20b8d73: Status 404 returned error can't find the container with id 8915529462f8c37f66480ecdc3bb261fc7f44dca4c5f49791c5d5b17f20b8d73
Sep 08 15:57:27.605809 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:27.605767    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf" event={"ID":"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540","Type":"ContainerStarted","Data":"8915529462f8c37f66480ecdc3bb261fc7f44dca4c5f49791c5d5b17f20b8d73"}
Sep 08 15:57:28.612929 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:28.612897    2552 generic.go:356] "Generic (PLEG): container finished" podID="f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540" containerID="f110967e448215a4d2a59ee1710b28f8375ac60405c1bb735904d3d8f7d00350" exitCode=0
Sep 08 15:57:28.613308 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:28.612987    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf" event={"ID":"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540","Type":"ContainerDied","Data":"f110967e448215a4d2a59ee1710b28f8375ac60405c1bb735904d3d8f7d00350"}
Sep 08 15:57:29.756734 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:29.756711    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"
Sep 08 15:57:29.808178 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:29.808140    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540-maas-api-service-ca\") pod \"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540\" (UID: \"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540\") "
Sep 08 15:57:29.808487 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:29.808462    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540" (UID: "f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:29.909517 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:29.909433    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:30.625211 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:30.625179    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"
Sep 08 15:57:30.625382 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:30.625218    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf" event={"ID":"f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540","Type":"ContainerDied","Data":"8915529462f8c37f66480ecdc3bb261fc7f44dca4c5f49791c5d5b17f20b8d73"}
Sep 08 15:57:30.625382 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:30.625277    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8915529462f8c37f66480ecdc3bb261fc7f44dca4c5f49791c5d5b17f20b8d73"
Sep 08 15:57:37.626946 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:37.626908    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"]
Sep 08 15:57:37.628903 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:37.628878    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-423c90-lzfdf"]
Sep 08 15:57:39.101352 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:39.101312    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540" path="/var/lib/kubelet/pods/f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540/volumes"
Sep 08 15:57:42.972786 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:42.972752    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"]
Sep 08 15:57:46.868243 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.868208    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"]
Sep 08 15:57:46.868671 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.868656    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540" containerName="revoke-keys"
Sep 08 15:57:46.868722 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.868672    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540" containerName="revoke-keys"
Sep 08 15:57:46.868776 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.868765    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a91e52-ae9d-45a8-ba0c-3a6a19f2e540" containerName="revoke-keys"
Sep 08 15:57:46.875645 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.872046    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"
Sep 08 15:57:46.879639 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.877509    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:57:46.879639 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.877848    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 15:57:46.882306 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.882280    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"]
Sep 08 15:57:46.885961 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.885599    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78820e4e-9761-45e8-a608-60ea938e530d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk\" (UID: \"78820e4e-9761-45e8-a608-60ea938e530d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"
Sep 08 15:57:46.986562 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.986524    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78820e4e-9761-45e8-a608-60ea938e530d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk\" (UID: \"78820e4e-9761-45e8-a608-60ea938e530d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"
Sep 08 15:57:46.987200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:46.987178    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78820e4e-9761-45e8-a608-60ea938e530d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk\" (UID: \"78820e4e-9761-45e8-a608-60ea938e530d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"
Sep 08 15:57:47.190450 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:47.190370    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"
Sep 08 15:57:47.475700 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:47.475661    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"]
Sep 08 15:57:47.477595 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:57:47.477559    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78820e4e_9761_45e8_a608_60ea938e530d.slice/crio-28da6cc9fc20198d93a622a68ebe65f9590522a587d7a871fea1bd1dc6d60af3 WatchSource:0}: Error finding container 28da6cc9fc20198d93a622a68ebe65f9590522a587d7a871fea1bd1dc6d60af3: Status 404 returned error can't find the container with id 28da6cc9fc20198d93a622a68ebe65f9590522a587d7a871fea1bd1dc6d60af3
Sep 08 15:57:47.718348 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:47.718309    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk" event={"ID":"78820e4e-9761-45e8-a608-60ea938e530d","Type":"ContainerStarted","Data":"28da6cc9fc20198d93a622a68ebe65f9590522a587d7a871fea1bd1dc6d60af3"}
Sep 08 15:57:48.726607 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:48.726569    2552 generic.go:356] "Generic (PLEG): container finished" podID="78820e4e-9761-45e8-a608-60ea938e530d" containerID="4f79e94f8b574575e44ebdae867e57bf3a89d5da74ebd47a77a3d8a5e367143f" exitCode=0
Sep 08 15:57:48.727004 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:48.726654    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk" event={"ID":"78820e4e-9761-45e8-a608-60ea938e530d","Type":"ContainerDied","Data":"4f79e94f8b574575e44ebdae867e57bf3a89d5da74ebd47a77a3d8a5e367143f"}
Sep 08 15:57:49.890722 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:49.890697    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"
Sep 08 15:57:49.914670 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:49.914641    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78820e4e-9761-45e8-a608-60ea938e530d-maas-api-service-ca\") pod \"78820e4e-9761-45e8-a608-60ea938e530d\" (UID: \"78820e4e-9761-45e8-a608-60ea938e530d\") "
Sep 08 15:57:49.914990 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:49.914964    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78820e4e-9761-45e8-a608-60ea938e530d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78820e4e-9761-45e8-a608-60ea938e530d" (UID: "78820e4e-9761-45e8-a608-60ea938e530d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:50.015771 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:50.015686    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78820e4e-9761-45e8-a608-60ea938e530d-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:50.740743 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:50.740712    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk" event={"ID":"78820e4e-9761-45e8-a608-60ea938e530d","Type":"ContainerDied","Data":"28da6cc9fc20198d93a622a68ebe65f9590522a587d7a871fea1bd1dc6d60af3"}
Sep 08 15:57:50.740743 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:50.740729    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"
Sep 08 15:57:50.740948 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:50.740755    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28da6cc9fc20198d93a622a68ebe65f9590522a587d7a871fea1bd1dc6d60af3"
Sep 08 15:57:53.020852 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:53.020818    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"]
Sep 08 15:57:53.023594 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:53.023566    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ac3191f1-6kdlk"]
Sep 08 15:57:53.103638 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:53.103594    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78820e4e-9761-45e8-a608-60ea938e530d" path="/var/lib/kubelet/pods/78820e4e-9761-45e8-a608-60ea938e530d/volumes"
Sep 08 15:57:57.993686 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:57.993627    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw" podUID="03d38e2e-236f-4df0-b262-fd23c8dd57d0" containerName="main" containerID="cri-o://f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7" gracePeriod=45
Sep 08 15:57:58.249566 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.249507    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:57:58.300422 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300385    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-dshm\") pod \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") "
Sep 08 15:57:58.300605 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300475    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-model-cache\") pod \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") "
Sep 08 15:57:58.300605 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300528    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tls-certs\") pod \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") "
Sep 08 15:57:58.300605 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300559    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tmp-dir\") pod \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") "
Sep 08 15:57:58.300605 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300600    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws2xz\" (UniqueName: \"kubernetes.io/projected/03d38e2e-236f-4df0-b262-fd23c8dd57d0-kube-api-access-ws2xz\") pod \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") "
Sep 08 15:57:58.300823 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300631    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-home\") pod \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\" (UID: \"03d38e2e-236f-4df0-b262-fd23c8dd57d0\") "
Sep 08 15:57:58.300823 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300760    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "03d38e2e-236f-4df0-b262-fd23c8dd57d0" (UID: "03d38e2e-236f-4df0-b262-fd23c8dd57d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:58.301011 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.300983    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "03d38e2e-236f-4df0-b262-fd23c8dd57d0" (UID: "03d38e2e-236f-4df0-b262-fd23c8dd57d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:58.301149 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.301122    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-home" (OuterVolumeSpecName: "home") pod "03d38e2e-236f-4df0-b262-fd23c8dd57d0" (UID: "03d38e2e-236f-4df0-b262-fd23c8dd57d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:58.301222 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.301180    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-model-cache\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:58.301275 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.301229    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tmp-dir\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:58.303105 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.303074    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-dshm" (OuterVolumeSpecName: "dshm") pod "03d38e2e-236f-4df0-b262-fd23c8dd57d0" (UID: "03d38e2e-236f-4df0-b262-fd23c8dd57d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:58.303195 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.303104    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d38e2e-236f-4df0-b262-fd23c8dd57d0-kube-api-access-ws2xz" (OuterVolumeSpecName: "kube-api-access-ws2xz") pod "03d38e2e-236f-4df0-b262-fd23c8dd57d0" (UID: "03d38e2e-236f-4df0-b262-fd23c8dd57d0"). InnerVolumeSpecName "kube-api-access-ws2xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:57:58.303195 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.303183    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "03d38e2e-236f-4df0-b262-fd23c8dd57d0" (UID: "03d38e2e-236f-4df0-b262-fd23c8dd57d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:57:58.402213 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.402173    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-dshm\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:58.402213 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.402205    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03d38e2e-236f-4df0-b262-fd23c8dd57d0-tls-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:58.402213 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.402216    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws2xz\" (UniqueName: \"kubernetes.io/projected/03d38e2e-236f-4df0-b262-fd23c8dd57d0-kube-api-access-ws2xz\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:58.402442 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.402226    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03d38e2e-236f-4df0-b262-fd23c8dd57d0-home\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:57:58.784494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.784456    2552 generic.go:356] "Generic (PLEG): container finished" podID="03d38e2e-236f-4df0-b262-fd23c8dd57d0" containerID="f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7" exitCode=0
Sep 08 15:57:58.784661 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.784501    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"
Sep 08 15:57:58.784661 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.784549    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw" event={"ID":"03d38e2e-236f-4df0-b262-fd23c8dd57d0","Type":"ContainerDied","Data":"f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7"}
Sep 08 15:57:58.784661 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.784598    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw" event={"ID":"03d38e2e-236f-4df0-b262-fd23c8dd57d0","Type":"ContainerDied","Data":"24df2f286b534d7885c18ea44c2819ebc4b97ef1fa07ba870b813493d8eb941d"}
Sep 08 15:57:58.784661 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.784617    2552 scope.go:117] "RemoveContainer" containerID="f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7"
Sep 08 15:57:58.794343 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.794325    2552 scope.go:117] "RemoveContainer" containerID="f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7"
Sep 08 15:57:58.794656 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:57:58.794632    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7\": container with ID starting with f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7 not found: ID does not exist" containerID="f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7"
Sep 08 15:57:58.794705 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.794665    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7"} err="failed to get container status \"f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7\": rpc error: code = NotFound desc = could not find container \"f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7\": container with ID starting with f9210de3e28eb5c5ad88af9584ae6adfc32604b0abf493a5ab02a57f9f9714f7 not found: ID does not exist"
Sep 08 15:57:58.809297 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.809270    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"]
Sep 08 15:57:58.813164 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:58.813142    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-423c90/autores-model-423c90-kserve-79fc774f75-dnpsw"]
Sep 08 15:57:59.102285 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:57:59.102209    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03d38e2e-236f-4df0-b262-fd23c8dd57d0" path="/var/lib/kubelet/pods/03d38e2e-236f-4df0-b262-fd23c8dd57d0/volumes"
Sep 08 15:58:22.238487 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.238404    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"]
Sep 08 15:58:22.241019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.238823    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03d38e2e-236f-4df0-b262-fd23c8dd57d0" containerName="main"
Sep 08 15:58:22.241019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.238835    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="03d38e2e-236f-4df0-b262-fd23c8dd57d0" containerName="main"
Sep 08 15:58:22.241019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.238851    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78820e4e-9761-45e8-a608-60ea938e530d" containerName="revoke-keys"
Sep 08 15:58:22.241019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.238857    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="78820e4e-9761-45e8-a608-60ea938e530d" containerName="revoke-keys"
Sep 08 15:58:22.241019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.238919    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="78820e4e-9761-45e8-a608-60ea938e530d" containerName="revoke-keys"
Sep 08 15:58:22.241019 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.238929    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="03d38e2e-236f-4df0-b262-fd23c8dd57d0" containerName="main"
Sep 08 15:58:22.241904 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.241889    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"
Sep 08 15:58:22.245279 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.245259    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:58:22.245492 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.245479    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 15:58:22.252868 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.252840    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"]
Sep 08 15:58:22.346592 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.346553    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a6ed16-8aad-4ad6-9c0a-5a385f2d5801-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9333348f-nq559\" (UID: \"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"
Sep 08 15:58:22.448200 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.448159    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a6ed16-8aad-4ad6-9c0a-5a385f2d5801-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9333348f-nq559\" (UID: \"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"
Sep 08 15:58:22.448841 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.448819    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a6ed16-8aad-4ad6-9c0a-5a385f2d5801-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9333348f-nq559\" (UID: \"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"
Sep 08 15:58:22.553928 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.553846    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"
Sep 08 15:58:22.662264 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.662228    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"]
Sep 08 15:58:22.828741 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.828706    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"]
Sep 08 15:58:22.830591 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:58:22.830564    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a6ed16_8aad_4ad6_9c0a_5a385f2d5801.slice/crio-3254d540a270d0a30515de9da5913f42ce4aad9034ae90b71c7c1839c7a9a52f WatchSource:0}: Error finding container 3254d540a270d0a30515de9da5913f42ce4aad9034ae90b71c7c1839c7a9a52f: Status 404 returned error can't find the container with id 3254d540a270d0a30515de9da5913f42ce4aad9034ae90b71c7c1839c7a9a52f
Sep 08 15:58:22.926972 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:22.926937    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559" event={"ID":"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801","Type":"ContainerStarted","Data":"3254d540a270d0a30515de9da5913f42ce4aad9034ae90b71c7c1839c7a9a52f"}
Sep 08 15:58:23.935068 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:23.935001    2552 generic.go:356] "Generic (PLEG): container finished" podID="08a6ed16-8aad-4ad6-9c0a-5a385f2d5801" containerID="29af0f895282a0ec7bbf0e780a2cf8119873c52699ec0a3b1f83e388dbac0945" exitCode=0
Sep 08 15:58:23.935494 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:23.935087    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559" event={"ID":"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801","Type":"ContainerDied","Data":"29af0f895282a0ec7bbf0e780a2cf8119873c52699ec0a3b1f83e388dbac0945"}
Sep 08 15:58:25.073313 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:25.073291    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"
Sep 08 15:58:25.176170 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:25.176134    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a6ed16-8aad-4ad6-9c0a-5a385f2d5801-maas-api-service-ca\") pod \"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801\" (UID: \"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801\") "
Sep 08 15:58:25.176532 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:25.176497    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08a6ed16-8aad-4ad6-9c0a-5a385f2d5801-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08a6ed16-8aad-4ad6-9c0a-5a385f2d5801" (UID: "08a6ed16-8aad-4ad6-9c0a-5a385f2d5801"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:58:25.277012 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:25.276936    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08a6ed16-8aad-4ad6-9c0a-5a385f2d5801-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:58:25.948994 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:25.948965    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"
Sep 08 15:58:25.949175 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:25.948965    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559" event={"ID":"08a6ed16-8aad-4ad6-9c0a-5a385f2d5801","Type":"ContainerDied","Data":"3254d540a270d0a30515de9da5913f42ce4aad9034ae90b71c7c1839c7a9a52f"}
Sep 08 15:58:25.949175 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:25.949078    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3254d540a270d0a30515de9da5913f42ce4aad9034ae90b71c7c1839c7a9a52f"
Sep 08 15:58:29.606655 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:29.606613    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"]
Sep 08 15:58:29.608497 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:29.608476    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9333348f-nq559"]
Sep 08 15:58:31.102451 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:31.102414    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08a6ed16-8aad-4ad6-9c0a-5a385f2d5801" path="/var/lib/kubelet/pods/08a6ed16-8aad-4ad6-9c0a-5a385f2d5801/volumes"
Sep 08 15:58:37.692489 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:37.692442    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr" podUID="b5868dad-11fa-44b5-b4e8-7ae6a30bea87" containerName="main" containerID="cri-o://65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75" gracePeriod=45
Sep 08 15:58:37.958363 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:37.958305    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:58:38.014721 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.014691    2552 generic.go:356] "Generic (PLEG): container finished" podID="b5868dad-11fa-44b5-b4e8-7ae6a30bea87" containerID="65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75" exitCode=0
Sep 08 15:58:38.014933 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.014763    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"
Sep 08 15:58:38.014933 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.014793    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr" event={"ID":"b5868dad-11fa-44b5-b4e8-7ae6a30bea87","Type":"ContainerDied","Data":"65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75"}
Sep 08 15:58:38.014933 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.014853    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr" event={"ID":"b5868dad-11fa-44b5-b4e8-7ae6a30bea87","Type":"ContainerDied","Data":"329667adefdacef1f6d1794f28d196924ac373039a3acf106a7a940f37da5a98"}
Sep 08 15:58:38.014933 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.014874    2552 scope.go:117] "RemoveContainer" containerID="65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75"
Sep 08 15:58:38.024446 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.024428    2552 scope.go:117] "RemoveContainer" containerID="65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75"
Sep 08 15:58:38.024779 ip-10-0-136-224 kubenswrapper[2552]: E0908 15:58:38.024759    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75\": container with ID starting with 65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75 not found: ID does not exist" containerID="65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75"
Sep 08 15:58:38.024833 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.024794    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75"} err="failed to get container status \"65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75\": rpc error: code = NotFound desc = could not find container \"65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75\": container with ID starting with 65a295235b21ea94e70ad1f432ab167a197d070c9e3ed6eaebe753f12d5f3b75 not found: ID does not exist"
Sep 08 15:58:38.113190 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113163    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-dshm\") pod \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") "
Sep 08 15:58:38.113190 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113205    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-model-cache\") pod \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") "
Sep 08 15:58:38.113443 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113266    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tls-certs\") pod \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") "
Sep 08 15:58:38.113443 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113290    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-home\") pod \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") "
Sep 08 15:58:38.113443 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113313    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tmp-dir\") pod \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") "
Sep 08 15:58:38.113443 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113353    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcb9r\" (UniqueName: \"kubernetes.io/projected/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-kube-api-access-dcb9r\") pod \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\" (UID: \"b5868dad-11fa-44b5-b4e8-7ae6a30bea87\") "
Sep 08 15:58:38.113640 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113563    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5868dad-11fa-44b5-b4e8-7ae6a30bea87" (UID: "b5868dad-11fa-44b5-b4e8-7ae6a30bea87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:38.113640 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113586    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-home" (OuterVolumeSpecName: "home") pod "b5868dad-11fa-44b5-b4e8-7ae6a30bea87" (UID: "b5868dad-11fa-44b5-b4e8-7ae6a30bea87"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:38.113640 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113597    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5868dad-11fa-44b5-b4e8-7ae6a30bea87" (UID: "b5868dad-11fa-44b5-b4e8-7ae6a30bea87"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:38.113789 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113738    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-model-cache\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:58:38.113789 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113751    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-home\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:58:38.113789 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.113761    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tmp-dir\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:58:38.115509 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.115479    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-dshm" (OuterVolumeSpecName: "dshm") pod "b5868dad-11fa-44b5-b4e8-7ae6a30bea87" (UID: "b5868dad-11fa-44b5-b4e8-7ae6a30bea87"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:38.115612 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.115531    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-kube-api-access-dcb9r" (OuterVolumeSpecName: "kube-api-access-dcb9r") pod "b5868dad-11fa-44b5-b4e8-7ae6a30bea87" (UID: "b5868dad-11fa-44b5-b4e8-7ae6a30bea87"). InnerVolumeSpecName "kube-api-access-dcb9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:58:38.115612 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.115583    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5868dad-11fa-44b5-b4e8-7ae6a30bea87" (UID: "b5868dad-11fa-44b5-b4e8-7ae6a30bea87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:58:38.215256 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.215163    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-tls-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:58:38.215256 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.215203    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcb9r\" (UniqueName: \"kubernetes.io/projected/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-kube-api-access-dcb9r\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:58:38.215256 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.215214    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5868dad-11fa-44b5-b4e8-7ae6a30bea87-dshm\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:58:38.339956 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.339925    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"]
Sep 08 15:58:38.341681 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:38.341645    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-e6f19b8d-kserve-56f4755658-66qzr"]
Sep 08 15:58:39.101300 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:58:39.101257    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5868dad-11fa-44b5-b4e8-7ae6a30bea87" path="/var/lib/kubelet/pods/b5868dad-11fa-44b5-b4e8-7ae6a30bea87/volumes"
Sep 08 15:59:22.692258 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.692225    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"]
Sep 08 15:59:22.692732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.692660    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5868dad-11fa-44b5-b4e8-7ae6a30bea87" containerName="main"
Sep 08 15:59:22.692732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.692678    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5868dad-11fa-44b5-b4e8-7ae6a30bea87" containerName="main"
Sep 08 15:59:22.692732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.692693    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a6ed16-8aad-4ad6-9c0a-5a385f2d5801" containerName="revoke-keys"
Sep 08 15:59:22.692732 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.692703    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a6ed16-8aad-4ad6-9c0a-5a385f2d5801" containerName="revoke-keys"
Sep 08 15:59:22.692869 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.692774    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a6ed16-8aad-4ad6-9c0a-5a385f2d5801" containerName="revoke-keys"
Sep 08 15:59:22.692869 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.692786    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5868dad-11fa-44b5-b4e8-7ae6a30bea87" containerName="main"
Sep 08 15:59:22.695965 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.695944    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.699666 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.699644    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c96e2b\"/\"test-model-c96e2b-kserve-self-signed-certs\""
Sep 08 15:59:22.699791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.699668    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c96e2b\"/\"default-dockercfg-pv6r4\""
Sep 08 15:59:22.699791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.699715    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c96e2b\"/\"openshift-service-ca.crt\""
Sep 08 15:59:22.699791 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.699668    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c96e2b\"/\"kube-root-ca.crt\""
Sep 08 15:59:22.705381 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.705357    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"]
Sep 08 15:59:22.850632 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.850600    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-model-cache\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.850842 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.850638    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-home\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.850842 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.850683    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tls-certs\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.850842 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.850760    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-dshm\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.850842 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.850794    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rvg2\" (UniqueName: \"kubernetes.io/projected/d0e830ec-3a8e-44af-88ee-75f681eaadc6-kube-api-access-6rvg2\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.851102 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.850851    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tmp-dir\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.951948 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.951860    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-model-cache\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.951948 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.951899    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-home\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.951948 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.951918    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tls-certs\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.952261 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.951964    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-dshm\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.952261 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.951990    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rvg2\" (UniqueName: \"kubernetes.io/projected/d0e830ec-3a8e-44af-88ee-75f681eaadc6-kube-api-access-6rvg2\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.952261 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.952019    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tmp-dir\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.952424 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.952384    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-home\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.952424 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.952394    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-model-cache\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.952519 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.952482    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tmp-dir\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.954337 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.954307    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-dshm\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.954561 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.954544    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tls-certs\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:22.960893 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:22.960870    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rvg2\" (UniqueName: \"kubernetes.io/projected/d0e830ec-3a8e-44af-88ee-75f681eaadc6-kube-api-access-6rvg2\") pod \"test-model-c96e2b-kserve-6fccdfb67f-jkn7t\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") " pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:23.006258 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:23.006223    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:23.305197 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:23.305165    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"]
Sep 08 15:59:23.306464 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:59:23.306438    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0e830ec_3a8e_44af_88ee_75f681eaadc6.slice/crio-b7ca809391b1217407c0a22887f8fced7c2421022d61e21056ba0f67cd2c7160 WatchSource:0}: Error finding container b7ca809391b1217407c0a22887f8fced7c2421022d61e21056ba0f67cd2c7160: Status 404 returned error can't find the container with id b7ca809391b1217407c0a22887f8fced7c2421022d61e21056ba0f67cd2c7160
Sep 08 15:59:24.266468 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:24.266433    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t" event={"ID":"d0e830ec-3a8e-44af-88ee-75f681eaadc6","Type":"ContainerStarted","Data":"fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2"}
Sep 08 15:59:24.266468 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:24.266478    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t" event={"ID":"d0e830ec-3a8e-44af-88ee-75f681eaadc6","Type":"ContainerStarted","Data":"b7ca809391b1217407c0a22887f8fced7c2421022d61e21056ba0f67cd2c7160"}
Sep 08 15:59:24.288654 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:24.288603    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t" podStartSLOduration=2.2885882779999998 podStartE2EDuration="2.288588278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:59:24.285893931 +0000 UTC m=+873.782301908" watchObservedRunningTime="2026-09-08 15:59:24.288588278 +0000 UTC m=+873.784996245"
Sep 08 15:59:33.006774 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:33.006733    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:33.007241 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:33.006793    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:33.019205 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:33.019183    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:33.324375 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:33.324291    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 15:59:39.234270 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.234221    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"]
Sep 08 15:59:39.238620 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.238591    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"
Sep 08 15:59:39.240767 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.240745    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:59:39.240911 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.240847    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 15:59:39.244978 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.244953    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"]
Sep 08 15:59:39.311176 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.311146    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c136a792-226b-4c08-a6f4-51f8498d8ca9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz\" (UID: \"c136a792-226b-4c08-a6f4-51f8498d8ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"
Sep 08 15:59:39.412568 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.412535    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c136a792-226b-4c08-a6f4-51f8498d8ca9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz\" (UID: \"c136a792-226b-4c08-a6f4-51f8498d8ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"
Sep 08 15:59:39.413217 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.413198    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c136a792-226b-4c08-a6f4-51f8498d8ca9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz\" (UID: \"c136a792-226b-4c08-a6f4-51f8498d8ca9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"
Sep 08 15:59:39.572758 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.572674    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"
Sep 08 15:59:39.847115 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:39.847085    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"]
Sep 08 15:59:39.848995 ip-10-0-136-224 kubenswrapper[2552]: W0908 15:59:39.848959    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc136a792_226b_4c08_a6f4_51f8498d8ca9.slice/crio-1e9c03f06ae810dbdb2896733d728bb6dfcc957aa561889695e3224488912254 WatchSource:0}: Error finding container 1e9c03f06ae810dbdb2896733d728bb6dfcc957aa561889695e3224488912254: Status 404 returned error can't find the container with id 1e9c03f06ae810dbdb2896733d728bb6dfcc957aa561889695e3224488912254
Sep 08 15:59:40.352141 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:40.352110    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz" event={"ID":"c136a792-226b-4c08-a6f4-51f8498d8ca9","Type":"ContainerStarted","Data":"243a36267abecee0b10025638fedc84e441596a849b975d80bd010ab12e0f4f8"}
Sep 08 15:59:40.352587 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:40.352172    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz" event={"ID":"c136a792-226b-4c08-a6f4-51f8498d8ca9","Type":"ContainerStarted","Data":"1e9c03f06ae810dbdb2896733d728bb6dfcc957aa561889695e3224488912254"}
Sep 08 15:59:40.367881 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:40.367837    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz" podStartSLOduration=1.367821056 podStartE2EDuration="1.367821056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:59:40.366406284 +0000 UTC m=+889.862814252" watchObservedRunningTime="2026-09-08 15:59:40.367821056 +0000 UTC m=+889.864229060"
Sep 08 15:59:41.359606 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:41.359563    2552 generic.go:356] "Generic (PLEG): container finished" podID="c136a792-226b-4c08-a6f4-51f8498d8ca9" containerID="243a36267abecee0b10025638fedc84e441596a849b975d80bd010ab12e0f4f8" exitCode=0
Sep 08 15:59:41.359975 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:41.359650    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz" event={"ID":"c136a792-226b-4c08-a6f4-51f8498d8ca9","Type":"ContainerDied","Data":"243a36267abecee0b10025638fedc84e441596a849b975d80bd010ab12e0f4f8"}
Sep 08 15:59:42.499407 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:42.499382    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"
Sep 08 15:59:42.543730 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:42.543701    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c136a792-226b-4c08-a6f4-51f8498d8ca9-maas-api-service-ca\") pod \"c136a792-226b-4c08-a6f4-51f8498d8ca9\" (UID: \"c136a792-226b-4c08-a6f4-51f8498d8ca9\") "
Sep 08 15:59:42.544071 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:42.544040    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c136a792-226b-4c08-a6f4-51f8498d8ca9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c136a792-226b-4c08-a6f4-51f8498d8ca9" (UID: "c136a792-226b-4c08-a6f4-51f8498d8ca9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:59:42.645231 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:42.645148    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c136a792-226b-4c08-a6f4-51f8498d8ca9-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 15:59:43.372568 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:43.372533    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"
Sep 08 15:59:43.372737 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:43.372530    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz" event={"ID":"c136a792-226b-4c08-a6f4-51f8498d8ca9","Type":"ContainerDied","Data":"1e9c03f06ae810dbdb2896733d728bb6dfcc957aa561889695e3224488912254"}
Sep 08 15:59:43.372737 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:43.372660    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e9c03f06ae810dbdb2896733d728bb6dfcc957aa561889695e3224488912254"
Sep 08 15:59:49.984687 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:49.984647    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"]
Sep 08 15:59:49.987186 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:49.987161    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b98e2-thmzz"]
Sep 08 15:59:51.102785 ip-10-0-136-224 kubenswrapper[2552]: I0908 15:59:51.102753    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c136a792-226b-4c08-a6f4-51f8498d8ca9" path="/var/lib/kubelet/pods/c136a792-226b-4c08-a6f4-51f8498d8ca9/volumes"
Sep 08 16:00:00.142317 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.142281    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg"]
Sep 08 16:00:00.142787 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.142689    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c136a792-226b-4c08-a6f4-51f8498d8ca9" containerName="revoke-keys"
Sep 08 16:00:00.142787 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.142700    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c136a792-226b-4c08-a6f4-51f8498d8ca9" containerName="revoke-keys"
Sep 08 16:00:00.142787 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.142772    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c136a792-226b-4c08-a6f4-51f8498d8ca9" containerName="revoke-keys"
Sep 08 16:00:00.146215 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.146199    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg"
Sep 08 16:00:00.149129 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.149109    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:00:00.156293 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.156267    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg"
Sep 08 16:00:00.159310 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.159282    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg"]
Sep 08 16:00:00.431540 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.431515    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg"]
Sep 08 16:00:00.433265 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:00:00.433234    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93653ae7_bf90_40ee_8b3b_4ef501943008.slice/crio-b7481584ed42a1b370bc99fec765bc93ce76eac05244debddb4869b559ccdf2b WatchSource:0}: Error finding container b7481584ed42a1b370bc99fec765bc93ce76eac05244debddb4869b559ccdf2b: Status 404 returned error can't find the container with id b7481584ed42a1b370bc99fec765bc93ce76eac05244debddb4869b559ccdf2b
Sep 08 16:00:00.471166 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:00.471133    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg" event={"ID":"93653ae7-bf90-40ee-8b3b-4ef501943008","Type":"ContainerStarted","Data":"b7481584ed42a1b370bc99fec765bc93ce76eac05244debddb4869b559ccdf2b"}
Sep 08 16:00:01.477793 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:01.477754    2552 generic.go:356] "Generic (PLEG): container finished" podID="93653ae7-bf90-40ee-8b3b-4ef501943008" containerID="465bbddd366a00ec52506bb27ad6c2312d5bcfeb51c5510057008fc3650d21fa" exitCode=0
Sep 08 16:00:01.478163 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:01.477836    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg" event={"ID":"93653ae7-bf90-40ee-8b3b-4ef501943008","Type":"ContainerDied","Data":"465bbddd366a00ec52506bb27ad6c2312d5bcfeb51c5510057008fc3650d21fa"}
Sep 08 16:00:02.621092 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:02.621067    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg"
Sep 08 16:00:03.489551 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:03.489522    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg"
Sep 08 16:00:03.489722 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:03.489527    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-qqspg" event={"ID":"93653ae7-bf90-40ee-8b3b-4ef501943008","Type":"ContainerDied","Data":"b7481584ed42a1b370bc99fec765bc93ce76eac05244debddb4869b559ccdf2b"}
Sep 08 16:00:03.489722 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:03.489644    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7481584ed42a1b370bc99fec765bc93ce76eac05244debddb4869b559ccdf2b"
Sep 08 16:00:32.341178 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.341134    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"]
Sep 08 16:00:32.456309 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.456276    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"]
Sep 08 16:00:32.456713 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.456701    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93653ae7-bf90-40ee-8b3b-4ef501943008" containerName="cleanup"
Sep 08 16:00:32.456761 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.456715    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="93653ae7-bf90-40ee-8b3b-4ef501943008" containerName="cleanup"
Sep 08 16:00:32.456795 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.456790    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="93653ae7-bf90-40ee-8b3b-4ef501943008" containerName="cleanup"
Sep 08 16:00:32.460238 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.460219    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"
Sep 08 16:00:32.462426 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.462399    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:00:32.462426 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.462414    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:00:32.468492 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.467917    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"]
Sep 08 16:00:32.640155 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.640110    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa171ca4-5abc-4bf6-9ee3-1238d59a95dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ef1190-djptp\" (UID: \"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"
Sep 08 16:00:32.740750 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.740717    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa171ca4-5abc-4bf6-9ee3-1238d59a95dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ef1190-djptp\" (UID: \"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"
Sep 08 16:00:32.741420 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.741399    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa171ca4-5abc-4bf6-9ee3-1238d59a95dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ef1190-djptp\" (UID: \"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"
Sep 08 16:00:32.772534 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:32.772511    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"
Sep 08 16:00:33.046190 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:33.046161    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"]
Sep 08 16:00:33.047804 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:00:33.047775    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa171ca4_5abc_4bf6_9ee3_1238d59a95dc.slice/crio-027a158bc67830b504bca8295766d8ad1ca3c9af32084724239562cbef2104f4 WatchSource:0}: Error finding container 027a158bc67830b504bca8295766d8ad1ca3c9af32084724239562cbef2104f4: Status 404 returned error can't find the container with id 027a158bc67830b504bca8295766d8ad1ca3c9af32084724239562cbef2104f4
Sep 08 16:00:33.668291 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:33.668256    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp" event={"ID":"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc","Type":"ContainerStarted","Data":"50ab601e6c7e9923e5968dd991681659f409f23e7ce8fa09eb504d343e207225"}
Sep 08 16:00:33.668656 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:33.668306    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp" event={"ID":"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc","Type":"ContainerStarted","Data":"027a158bc67830b504bca8295766d8ad1ca3c9af32084724239562cbef2104f4"}
Sep 08 16:00:33.684284 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:33.684233    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp" podStartSLOduration=1.684217972 podStartE2EDuration="1.684217972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:00:33.682821847 +0000 UTC m=+943.179229814" watchObservedRunningTime="2026-09-08 16:00:33.684217972 +0000 UTC m=+943.180625938"
Sep 08 16:00:34.675444 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:34.675409    2552 generic.go:356] "Generic (PLEG): container finished" podID="fa171ca4-5abc-4bf6-9ee3-1238d59a95dc" containerID="50ab601e6c7e9923e5968dd991681659f409f23e7ce8fa09eb504d343e207225" exitCode=0
Sep 08 16:00:34.675853 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:34.675501    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp" event={"ID":"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc","Type":"ContainerDied","Data":"50ab601e6c7e9923e5968dd991681659f409f23e7ce8fa09eb504d343e207225"}
Sep 08 16:00:35.820888 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:35.820863    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"
Sep 08 16:00:35.975556 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:35.975469    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa171ca4-5abc-4bf6-9ee3-1238d59a95dc-maas-api-service-ca\") pod \"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc\" (UID: \"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc\") "
Sep 08 16:00:35.975843 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:35.975819    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa171ca4-5abc-4bf6-9ee3-1238d59a95dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa171ca4-5abc-4bf6-9ee3-1238d59a95dc" (UID: "fa171ca4-5abc-4bf6-9ee3-1238d59a95dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:00:36.077105 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:36.077073    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa171ca4-5abc-4bf6-9ee3-1238d59a95dc-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:36.689465 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:36.689436    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"
Sep 08 16:00:36.689636 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:36.689477    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp" event={"ID":"fa171ca4-5abc-4bf6-9ee3-1238d59a95dc","Type":"ContainerDied","Data":"027a158bc67830b504bca8295766d8ad1ca3c9af32084724239562cbef2104f4"}
Sep 08 16:00:36.689636 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:36.689524    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="027a158bc67830b504bca8295766d8ad1ca3c9af32084724239562cbef2104f4"
Sep 08 16:00:43.475470 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:43.475436    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"]
Sep 08 16:00:43.478066 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:43.478042    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ef1190-djptp"]
Sep 08 16:00:44.539989 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.539948    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"]
Sep 08 16:00:44.540507 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.540392    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa171ca4-5abc-4bf6-9ee3-1238d59a95dc" containerName="revoke-keys"
Sep 08 16:00:44.540507 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.540406    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa171ca4-5abc-4bf6-9ee3-1238d59a95dc" containerName="revoke-keys"
Sep 08 16:00:44.540507 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.540484    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa171ca4-5abc-4bf6-9ee3-1238d59a95dc" containerName="revoke-keys"
Sep 08 16:00:44.543779 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.543755    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"
Sep 08 16:00:44.545914 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.545890    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:00:44.546040 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.545918    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:00:44.549068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.549017    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/444ccecd-05ba-4762-a950-888a89685237-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf\" (UID: \"444ccecd-05ba-4762-a950-888a89685237\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"
Sep 08 16:00:44.552123 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.551979    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"]
Sep 08 16:00:44.649802 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.649766    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/444ccecd-05ba-4762-a950-888a89685237-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf\" (UID: \"444ccecd-05ba-4762-a950-888a89685237\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"
Sep 08 16:00:44.650506 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.650481    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/444ccecd-05ba-4762-a950-888a89685237-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf\" (UID: \"444ccecd-05ba-4762-a950-888a89685237\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"
Sep 08 16:00:44.856225 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:44.856142    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"
Sep 08 16:00:45.101314 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:45.101281    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa171ca4-5abc-4bf6-9ee3-1238d59a95dc" path="/var/lib/kubelet/pods/fa171ca4-5abc-4bf6-9ee3-1238d59a95dc/volumes"
Sep 08 16:00:45.127270 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:45.127245    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"]
Sep 08 16:00:45.129420 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:00:45.129393    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444ccecd_05ba_4762_a950_888a89685237.slice/crio-18b0d4d493c0197917f2348064ad0fe411fcc5e3ac63536540879b704025f82d WatchSource:0}: Error finding container 18b0d4d493c0197917f2348064ad0fe411fcc5e3ac63536540879b704025f82d: Status 404 returned error can't find the container with id 18b0d4d493c0197917f2348064ad0fe411fcc5e3ac63536540879b704025f82d
Sep 08 16:00:45.742172 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:45.742077    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf" event={"ID":"444ccecd-05ba-4762-a950-888a89685237","Type":"ContainerStarted","Data":"45f0a6d176255d4bce0b27864afe112bc01e4aeb910cb9d274fc4b0128d82153"}
Sep 08 16:00:45.742172 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:45.742134    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf" event={"ID":"444ccecd-05ba-4762-a950-888a89685237","Type":"ContainerStarted","Data":"18b0d4d493c0197917f2348064ad0fe411fcc5e3ac63536540879b704025f82d"}
Sep 08 16:00:45.777705 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:45.777648    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf" podStartSLOduration=1.777631837 podStartE2EDuration="1.777631837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:00:45.774676976 +0000 UTC m=+955.271084942" watchObservedRunningTime="2026-09-08 16:00:45.777631837 +0000 UTC m=+955.274039803"
Sep 08 16:00:46.749689 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:46.749657    2552 generic.go:356] "Generic (PLEG): container finished" podID="444ccecd-05ba-4762-a950-888a89685237" containerID="45f0a6d176255d4bce0b27864afe112bc01e4aeb910cb9d274fc4b0128d82153" exitCode=0
Sep 08 16:00:46.750086 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:46.749744    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf" event={"ID":"444ccecd-05ba-4762-a950-888a89685237","Type":"ContainerDied","Data":"45f0a6d176255d4bce0b27864afe112bc01e4aeb910cb9d274fc4b0128d82153"}
Sep 08 16:00:47.361443 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.361400    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t" podUID="d0e830ec-3a8e-44af-88ee-75f681eaadc6" containerName="main" containerID="cri-o://fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2" gracePeriod=45
Sep 08 16:00:47.619385 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.619318    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 16:00:47.679235 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679200    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-dshm\") pod \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") "
Sep 08 16:00:47.679235 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679236    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tmp-dir\") pod \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") "
Sep 08 16:00:47.679437 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679260    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tls-certs\") pod \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") "
Sep 08 16:00:47.679437 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679285    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-model-cache\") pod \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") "
Sep 08 16:00:47.679437 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679362    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-home\") pod \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") "
Sep 08 16:00:47.679437 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679422    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rvg2\" (UniqueName: \"kubernetes.io/projected/d0e830ec-3a8e-44af-88ee-75f681eaadc6-kube-api-access-6rvg2\") pod \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\" (UID: \"d0e830ec-3a8e-44af-88ee-75f681eaadc6\") "
Sep 08 16:00:47.679648 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679499    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0e830ec-3a8e-44af-88ee-75f681eaadc6" (UID: "d0e830ec-3a8e-44af-88ee-75f681eaadc6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:00:47.679648 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679531    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0e830ec-3a8e-44af-88ee-75f681eaadc6" (UID: "d0e830ec-3a8e-44af-88ee-75f681eaadc6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:00:47.679745 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679673    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-home" (OuterVolumeSpecName: "home") pod "d0e830ec-3a8e-44af-88ee-75f681eaadc6" (UID: "d0e830ec-3a8e-44af-88ee-75f681eaadc6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:00:47.679817 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679795    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tmp-dir\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:47.679817 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679822    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-model-cache\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:47.680004 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.679831    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-home\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:47.681583 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.681563    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-dshm" (OuterVolumeSpecName: "dshm") pod "d0e830ec-3a8e-44af-88ee-75f681eaadc6" (UID: "d0e830ec-3a8e-44af-88ee-75f681eaadc6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:00:47.681671 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.681603    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e830ec-3a8e-44af-88ee-75f681eaadc6-kube-api-access-6rvg2" (OuterVolumeSpecName: "kube-api-access-6rvg2") pod "d0e830ec-3a8e-44af-88ee-75f681eaadc6" (UID: "d0e830ec-3a8e-44af-88ee-75f681eaadc6"). InnerVolumeSpecName "kube-api-access-6rvg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:00:47.681671 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.681620    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0e830ec-3a8e-44af-88ee-75f681eaadc6" (UID: "d0e830ec-3a8e-44af-88ee-75f681eaadc6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:00:47.762510 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.762469    2552 generic.go:356] "Generic (PLEG): container finished" podID="d0e830ec-3a8e-44af-88ee-75f681eaadc6" containerID="fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2" exitCode=0
Sep 08 16:00:47.762966 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.762543    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"
Sep 08 16:00:47.762966 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.762553    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t" event={"ID":"d0e830ec-3a8e-44af-88ee-75f681eaadc6","Type":"ContainerDied","Data":"fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2"}
Sep 08 16:00:47.762966 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.762601    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t" event={"ID":"d0e830ec-3a8e-44af-88ee-75f681eaadc6","Type":"ContainerDied","Data":"b7ca809391b1217407c0a22887f8fced7c2421022d61e21056ba0f67cd2c7160"}
Sep 08 16:00:47.762966 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.762636    2552 scope.go:117] "RemoveContainer" containerID="fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2"
Sep 08 16:00:47.772972 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.772951    2552 scope.go:117] "RemoveContainer" containerID="fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2"
Sep 08 16:00:47.773321 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:00:47.773294    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2\": container with ID starting with fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2 not found: ID does not exist" containerID="fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2"
Sep 08 16:00:47.773405 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.773332    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2"} err="failed to get container status \"fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2\": rpc error: code = NotFound desc = could not find container \"fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2\": container with ID starting with fe680fdf5c2e09a1c6ca05955a35e62b6e0ad72f906d350587123fab9f8d85e2 not found: ID does not exist"
Sep 08 16:00:47.780475 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.780454    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rvg2\" (UniqueName: \"kubernetes.io/projected/d0e830ec-3a8e-44af-88ee-75f681eaadc6-kube-api-access-6rvg2\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:47.780475 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.780475    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0e830ec-3a8e-44af-88ee-75f681eaadc6-dshm\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:47.780609 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.780485    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0e830ec-3a8e-44af-88ee-75f681eaadc6-tls-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:47.786273 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.786249    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"]
Sep 08 16:00:47.789387 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.789368    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c96e2b/test-model-c96e2b-kserve-6fccdfb67f-jkn7t"]
Sep 08 16:00:47.884529 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.884507    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"
Sep 08 16:00:47.982362 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.982332    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/444ccecd-05ba-4762-a950-888a89685237-maas-api-service-ca\") pod \"444ccecd-05ba-4762-a950-888a89685237\" (UID: \"444ccecd-05ba-4762-a950-888a89685237\") "
Sep 08 16:00:47.982700 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:47.982677    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/444ccecd-05ba-4762-a950-888a89685237-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "444ccecd-05ba-4762-a950-888a89685237" (UID: "444ccecd-05ba-4762-a950-888a89685237"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:00:48.083742 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:48.083709    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/444ccecd-05ba-4762-a950-888a89685237-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:48.769918 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:48.769831    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf" event={"ID":"444ccecd-05ba-4762-a950-888a89685237","Type":"ContainerDied","Data":"18b0d4d493c0197917f2348064ad0fe411fcc5e3ac63536540879b704025f82d"}
Sep 08 16:00:48.769918 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:48.769852    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"
Sep 08 16:00:48.769918 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:48.769880    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18b0d4d493c0197917f2348064ad0fe411fcc5e3ac63536540879b704025f82d"
Sep 08 16:00:49.101725 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:49.101640    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e830ec-3a8e-44af-88ee-75f681eaadc6" path="/var/lib/kubelet/pods/d0e830ec-3a8e-44af-88ee-75f681eaadc6/volumes"
Sep 08 16:00:50.188465 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.188430    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"]
Sep 08 16:00:50.188883 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.188869    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0e830ec-3a8e-44af-88ee-75f681eaadc6" containerName="main"
Sep 08 16:00:50.188932 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.188888    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e830ec-3a8e-44af-88ee-75f681eaadc6" containerName="main"
Sep 08 16:00:50.188932 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.188902    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444ccecd-05ba-4762-a950-888a89685237" containerName="revoke-keys"
Sep 08 16:00:50.188932 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.188908    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="444ccecd-05ba-4762-a950-888a89685237" containerName="revoke-keys"
Sep 08 16:00:50.189055 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.188986    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0e830ec-3a8e-44af-88ee-75f681eaadc6" containerName="main"
Sep 08 16:00:50.189055 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.188995    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="444ccecd-05ba-4762-a950-888a89685237" containerName="revoke-keys"
Sep 08 16:00:50.192343 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.192320    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"
Sep 08 16:00:50.194441 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.194419    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:00:50.194600 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.194579    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:00:50.198772 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.198737    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"]
Sep 08 16:00:50.306531 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.306493    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c80ff4-f236-401a-a4ed-3ba446223bd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb\" (UID: \"b0c80ff4-f236-401a-a4ed-3ba446223bd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"
Sep 08 16:00:50.407690 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.407652    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c80ff4-f236-401a-a4ed-3ba446223bd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb\" (UID: \"b0c80ff4-f236-401a-a4ed-3ba446223bd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"
Sep 08 16:00:50.408298 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.408280    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c80ff4-f236-401a-a4ed-3ba446223bd4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb\" (UID: \"b0c80ff4-f236-401a-a4ed-3ba446223bd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"
Sep 08 16:00:50.506699 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.506623    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"
Sep 08 16:00:50.785766 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:50.785741    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"]
Sep 08 16:00:50.786961 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:00:50.786927    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c80ff4_f236_401a_a4ed_3ba446223bd4.slice/crio-8191f192b1eb08f46085cae9ac97082e0909a94f17e6060777e6a3e2d326f1a2 WatchSource:0}: Error finding container 8191f192b1eb08f46085cae9ac97082e0909a94f17e6060777e6a3e2d326f1a2: Status 404 returned error can't find the container with id 8191f192b1eb08f46085cae9ac97082e0909a94f17e6060777e6a3e2d326f1a2
Sep 08 16:00:51.789932 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:51.789900    2552 generic.go:356] "Generic (PLEG): container finished" podID="b0c80ff4-f236-401a-a4ed-3ba446223bd4" containerID="017fb4c70db05d0cc58e24b1a2a7fdeedb5b6aee8cab254c79232d2135fc7e58" exitCode=0
Sep 08 16:00:51.790263 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:51.789986    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb" event={"ID":"b0c80ff4-f236-401a-a4ed-3ba446223bd4","Type":"ContainerDied","Data":"017fb4c70db05d0cc58e24b1a2a7fdeedb5b6aee8cab254c79232d2135fc7e58"}
Sep 08 16:00:51.790263 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:51.790054    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb" event={"ID":"b0c80ff4-f236-401a-a4ed-3ba446223bd4","Type":"ContainerStarted","Data":"8191f192b1eb08f46085cae9ac97082e0909a94f17e6060777e6a3e2d326f1a2"}
Sep 08 16:00:52.925057 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:52.925003    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"
Sep 08 16:00:53.035644 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:53.035613    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c80ff4-f236-401a-a4ed-3ba446223bd4-maas-api-service-ca\") pod \"b0c80ff4-f236-401a-a4ed-3ba446223bd4\" (UID: \"b0c80ff4-f236-401a-a4ed-3ba446223bd4\") "
Sep 08 16:00:53.035986 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:53.035964    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0c80ff4-f236-401a-a4ed-3ba446223bd4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0c80ff4-f236-401a-a4ed-3ba446223bd4" (UID: "b0c80ff4-f236-401a-a4ed-3ba446223bd4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:00:53.136823 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:53.136791    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0c80ff4-f236-401a-a4ed-3ba446223bd4-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:00:53.801610 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:53.801581    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb" event={"ID":"b0c80ff4-f236-401a-a4ed-3ba446223bd4","Type":"ContainerDied","Data":"8191f192b1eb08f46085cae9ac97082e0909a94f17e6060777e6a3e2d326f1a2"}
Sep 08 16:00:53.801610 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:53.801602    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"
Sep 08 16:00:53.801868 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:53.801626    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8191f192b1eb08f46085cae9ac97082e0909a94f17e6060777e6a3e2d326f1a2"
Sep 08 16:00:55.603919 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:55.603884    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"]
Sep 08 16:00:55.606688 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:55.606662    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cf23f7aa-q6wnf"]
Sep 08 16:00:57.101361 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:00:57.101328    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="444ccecd-05ba-4762-a950-888a89685237" path="/var/lib/kubelet/pods/444ccecd-05ba-4762-a950-888a89685237/volumes"
Sep 08 16:01:00.990868 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:00.990836    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"]
Sep 08 16:01:00.991385 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:00.991066    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g" podUID="84cdf92a-a938-4b95-b6b2-b45fa3c7659c" containerName="payload-processing" containerID="cri-o://71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d" gracePeriod=30
Sep 08 16:01:00.997632 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:00.997600    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"]
Sep 08 16:01:00.997883 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:00.997858    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd" podUID="00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" containerName="payload-pre-processing" containerID="cri-o://ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99" gracePeriod=30
Sep 08 16:01:01.151692 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.151659    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"]
Sep 08 16:01:01.153723 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.153701    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c96e2b-ggqhb"]
Sep 08 16:01:01.280967 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.280945    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 16:01:01.283772 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.283755    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 16:01:01.418553 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.418517    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkzns\" (UniqueName: \"kubernetes.io/projected/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-kube-api-access-wkzns\") pod \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") "
Sep 08 16:01:01.418743 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.418571    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-plugins-config-volume\") pod \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\" (UID: \"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192\") "
Sep 08 16:01:01.418743 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.418596    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-plugins-config-volume\") pod \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") "
Sep 08 16:01:01.418743 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.418677    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl8nc\" (UniqueName: \"kubernetes.io/projected/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-kube-api-access-sl8nc\") pod \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\" (UID: \"84cdf92a-a938-4b95-b6b2-b45fa3c7659c\") "
Sep 08 16:01:01.418984 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.418953    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" (UID: "00cea2d1-13f2-4b08-a0fc-4eefcbd3f192"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:01:01.419512 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.419476    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84cdf92a-a938-4b95-b6b2-b45fa3c7659c" (UID: "84cdf92a-a938-4b95-b6b2-b45fa3c7659c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:01:01.420897 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.420860    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-kube-api-access-wkzns" (OuterVolumeSpecName: "kube-api-access-wkzns") pod "00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" (UID: "00cea2d1-13f2-4b08-a0fc-4eefcbd3f192"). InnerVolumeSpecName "kube-api-access-wkzns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:01:01.423066 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.421592    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-kube-api-access-sl8nc" (OuterVolumeSpecName: "kube-api-access-sl8nc") pod "84cdf92a-a938-4b95-b6b2-b45fa3c7659c" (UID: "84cdf92a-a938-4b95-b6b2-b45fa3c7659c"). InnerVolumeSpecName "kube-api-access-sl8nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:01:01.520056 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.519941    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl8nc\" (UniqueName: \"kubernetes.io/projected/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-kube-api-access-sl8nc\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:01:01.520056 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.519971    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkzns\" (UniqueName: \"kubernetes.io/projected/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-kube-api-access-wkzns\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:01:01.520056 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.519980    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:01:01.520056 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.519989    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84cdf92a-a938-4b95-b6b2-b45fa3c7659c-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:01:01.847641 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.847553    2552 generic.go:356] "Generic (PLEG): container finished" podID="84cdf92a-a938-4b95-b6b2-b45fa3c7659c" containerID="71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d" exitCode=0
Sep 08 16:01:01.847641 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.847623    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"
Sep 08 16:01:01.847875 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.847634    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g" event={"ID":"84cdf92a-a938-4b95-b6b2-b45fa3c7659c","Type":"ContainerDied","Data":"71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d"}
Sep 08 16:01:01.847875 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.847719    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g" event={"ID":"84cdf92a-a938-4b95-b6b2-b45fa3c7659c","Type":"ContainerDied","Data":"5f33998e3774c5532086b9c30b137e29906a54e14ce5cd326e322dad2cae8105"}
Sep 08 16:01:01.847875 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.847751    2552 scope.go:117] "RemoveContainer" containerID="71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d"
Sep 08 16:01:01.849008 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.848987    2552 generic.go:356] "Generic (PLEG): container finished" podID="00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" containerID="ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99" exitCode=0
Sep 08 16:01:01.849097 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.849053    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"
Sep 08 16:01:01.849097 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.849064    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd" event={"ID":"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192","Type":"ContainerDied","Data":"ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99"}
Sep 08 16:01:01.849179 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.849117    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd" event={"ID":"00cea2d1-13f2-4b08-a0fc-4eefcbd3f192","Type":"ContainerDied","Data":"e492b1fcfcd1baf134f8ccad814008f39d8a84f89090866d3a845fa7a6f47f41"}
Sep 08 16:01:01.858067 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.858048    2552 scope.go:117] "RemoveContainer" containerID="71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d"
Sep 08 16:01:01.858358 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:01:01.858340    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d\": container with ID starting with 71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d not found: ID does not exist" containerID="71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d"
Sep 08 16:01:01.858432 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.858364    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d"} err="failed to get container status \"71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d\": rpc error: code = NotFound desc = could not find container \"71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d\": container with ID starting with 71ee4c532ffe4737c7f498af940c5d93db41346a0424db58c4ac461cb517f31d not found: ID does not exist"
Sep 08 16:01:01.858432 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.858380    2552 scope.go:117] "RemoveContainer" containerID="ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99"
Sep 08 16:01:01.866879 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.866860    2552 scope.go:117] "RemoveContainer" containerID="ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99"
Sep 08 16:01:01.867151 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:01:01.867129    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99\": container with ID starting with ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99 not found: ID does not exist" containerID="ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99"
Sep 08 16:01:01.867218 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.867157    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99"} err="failed to get container status \"ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99\": rpc error: code = NotFound desc = could not find container \"ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99\": container with ID starting with ece344599dc3e377f08e0ad45ab377db9b5bdc19239d85d96c1bc6459d5ade99 not found: ID does not exist"
Sep 08 16:01:01.872873 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.872851    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"]
Sep 08 16:01:01.876478 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.876457    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c96e2b-5f4756fc66-gkdsd"]
Sep 08 16:01:01.885994 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.885971    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"]
Sep 08 16:01:01.889414 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:01.889393    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c96e2b-69497489d9-xlr5g"]
Sep 08 16:01:03.101392 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:03.101362    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" path="/var/lib/kubelet/pods/00cea2d1-13f2-4b08-a0fc-4eefcbd3f192/volumes"
Sep 08 16:01:03.101859 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:03.101843    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84cdf92a-a938-4b95-b6b2-b45fa3c7659c" path="/var/lib/kubelet/pods/84cdf92a-a938-4b95-b6b2-b45fa3c7659c/volumes"
Sep 08 16:01:03.102186 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:03.102173    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0c80ff4-f236-401a-a4ed-3ba446223bd4" path="/var/lib/kubelet/pods/b0c80ff4-f236-401a-a4ed-3ba446223bd4/volumes"
Sep 08 16:01:45.353190 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353155    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"]
Sep 08 16:01:45.353642 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353627    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84cdf92a-a938-4b95-b6b2-b45fa3c7659c" containerName="payload-processing"
Sep 08 16:01:45.353687 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353644    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="84cdf92a-a938-4b95-b6b2-b45fa3c7659c" containerName="payload-processing"
Sep 08 16:01:45.353687 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353655    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" containerName="payload-pre-processing"
Sep 08 16:01:45.353687 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353661    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" containerName="payload-pre-processing"
Sep 08 16:01:45.353687 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353671    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0c80ff4-f236-401a-a4ed-3ba446223bd4" containerName="revoke-keys"
Sep 08 16:01:45.353687 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353676    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0c80ff4-f236-401a-a4ed-3ba446223bd4" containerName="revoke-keys"
Sep 08 16:01:45.353833 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353731    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="00cea2d1-13f2-4b08-a0fc-4eefcbd3f192" containerName="payload-pre-processing"
Sep 08 16:01:45.353833 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353739    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0c80ff4-f236-401a-a4ed-3ba446223bd4" containerName="revoke-keys"
Sep 08 16:01:45.353833 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.353749    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="84cdf92a-a938-4b95-b6b2-b45fa3c7659c" containerName="payload-processing"
Sep 08 16:01:45.357727 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.357704    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.360045 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.360003    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-e69292-openshift-default-dockercfg-bvddv\""
Sep 08 16:01:45.373648 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.373621    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"]
Sep 08 16:01:45.467055 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.466995    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-socket\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467055 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467053    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-token\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467464 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467134    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f3e7e73-3454-42d9-a380-8e47787bee59-istiod-ca-cert\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467464 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467172    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-credential-socket\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467464 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467231    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-data\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467464 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467256    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6gsd\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-kube-api-access-g6gsd\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467464 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467341    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-certs\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467464 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467394    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-envoy\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.467464 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.467423    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-podinfo\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568036 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.567998    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-certs\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568212 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568061    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-envoy\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568212 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568088    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-podinfo\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568212 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568119    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-socket\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568212 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568145    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-token\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568212 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568202    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f3e7e73-3454-42d9-a380-8e47787bee59-istiod-ca-cert\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568492 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568225    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-credential-socket\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568492 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568277    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-data\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568492 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568299    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gsd\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-kube-api-access-g6gsd\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568492 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568417    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-certs\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568743 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568505    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-socket\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568743 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568686    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-credential-socket\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568875 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568851    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-data\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.568940 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.568920    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f3e7e73-3454-42d9-a380-8e47787bee59-istiod-ca-cert\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.570659 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.570625    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-envoy\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.570771 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.570715    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-podinfo\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.575707 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.575686    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-token\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.575823 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.575764    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6gsd\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-kube-api-access-g6gsd\") pod \"e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") " pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.669434 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.669402    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:45.948388 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.948362    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"]
Sep 08 16:01:45.950501 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:01:45.950472    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3e7e73_3454_42d9_a380_8e47787bee59.slice/crio-f2193df1d028d1aee325160fba63eaa50af392beb5949027ec4bc911a252f2d3 WatchSource:0}: Error finding container f2193df1d028d1aee325160fba63eaa50af392beb5949027ec4bc911a252f2d3: Status 404 returned error can't find the container with id f2193df1d028d1aee325160fba63eaa50af392beb5949027ec4bc911a252f2d3
Sep 08 16:01:45.952712 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.952682    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:01:45.952776 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.952748    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:01:45.952814 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:45.952776    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:01:46.099656 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:46.099608    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" event={"ID":"3f3e7e73-3454-42d9-a380-8e47787bee59","Type":"ContainerStarted","Data":"26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442"}
Sep 08 16:01:46.099813 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:46.099675    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" event={"ID":"3f3e7e73-3454-42d9-a380-8e47787bee59","Type":"ContainerStarted","Data":"f2193df1d028d1aee325160fba63eaa50af392beb5949027ec4bc911a252f2d3"}
Sep 08 16:01:46.123174 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:46.123109    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podStartSLOduration=1.123085016 podStartE2EDuration="1.123085016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:01:46.119550124 +0000 UTC m=+1015.615958092" watchObservedRunningTime="2026-09-08 16:01:46.123085016 +0000 UTC m=+1015.619492984"
Sep 08 16:01:46.670369 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:46.670336    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:47.674992 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:47.674958    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:48.110513 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:48.110492    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:01:48.111399 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:01:48.111380    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:02:03.535119 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:03.535077    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:41734->10.134.0.50:15021: read: connection reset by peer" start-of-body=
Sep 08 16:02:03.535585 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:03.535155    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:41734->10.134.0.50:15021: read: connection reset by peer"
Sep 08 16:02:04.212657 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:04.212616    2552 generic.go:356] "Generic (PLEG): container finished" podID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerID="26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442" exitCode=0
Sep 08 16:02:04.212866 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:04.212665    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" event={"ID":"3f3e7e73-3454-42d9-a380-8e47787bee59","Type":"ContainerDied","Data":"26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442"}
Sep 08 16:02:04.213197 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:04.213176    2552 scope.go:117] "RemoveContainer" containerID="26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442"
Sep 08 16:02:04.214322 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:04.214293    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:02:04.214420 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:04.214379    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:02:04.214483 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:04.214427    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:02:05.220461 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:05.220424    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" event={"ID":"3f3e7e73-3454-42d9-a380-8e47787bee59","Type":"ContainerStarted","Data":"e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e"}
Sep 08 16:02:05.670082 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:05.670017    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:02:05.670321 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:05.670263    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 16:02:05.670364 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:05.670320    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 16:02:06.670753 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:06.670718    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 16:02:06.671156 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:06.670781    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 16:02:07.670387 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:07.670349    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 16:02:07.670562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:07.670409    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 16:02:08.670841 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:08.670805    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 16:02:08.671266 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:08.670870    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 16:02:09.670806 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:09.670765    2552 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 08 16:02:09.670994 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:09.670878    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 08 16:02:10.674312 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:10.674274    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:02:10.674676 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:10.674524    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:02:10.675187 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:10.675165    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:02:11.770216 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.770187    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"]
Sep 08 16:02:11.773110 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.773089    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.775374 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.775348    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-108124\"/\"kube-root-ca.crt\""
Sep 08 16:02:11.776107 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.776090    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-108124\"/\"default-dockercfg-rch8r\""
Sep 08 16:02:11.776211 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.776130    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-108124\"/\"openshift-service-ca.crt\""
Sep 08 16:02:11.776309 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.776281    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-108124\"/\"auth-test-model-108124-kserve-self-signed-certs\""
Sep 08 16:02:11.784056 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.784000    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"]
Sep 08 16:02:11.836720 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.836687    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e692b4-834e-4fe6-9b3d-e5976560d958-tls-certs\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.836917 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.836732    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-dshm\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.836917 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.836781    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs552\" (UniqueName: \"kubernetes.io/projected/26e692b4-834e-4fe6-9b3d-e5976560d958-kube-api-access-bs552\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.836917 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.836876    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-tmp-dir\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.836917 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.836910    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-home\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.837142 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.836935    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-model-cache\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.937835 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.937794    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e692b4-834e-4fe6-9b3d-e5976560d958-tls-certs\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.937835 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.937835    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-dshm\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.938070 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.937864    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs552\" (UniqueName: \"kubernetes.io/projected/26e692b4-834e-4fe6-9b3d-e5976560d958-kube-api-access-bs552\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.938070 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.937915    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-tmp-dir\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.938070 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.937937    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-home\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.938221 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.938066    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-model-cache\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.938429 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.938405    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-tmp-dir\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.938515 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.938429    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-home\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.938515 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.938461    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-model-cache\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.939996 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.939973    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-dshm\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.940283 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.940263    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e692b4-834e-4fe6-9b3d-e5976560d958-tls-certs\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:11.945036 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:11.944999    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs552\" (UniqueName: \"kubernetes.io/projected/26e692b4-834e-4fe6-9b3d-e5976560d958-kube-api-access-bs552\") pod \"auth-test-model-108124-kserve-795bd74799-p8sfn\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") " pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:12.083590 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:12.083493    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:12.361551 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:12.361519    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"]
Sep 08 16:02:12.363350 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:02:12.363323    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26e692b4_834e_4fe6_9b3d_e5976560d958.slice/crio-3aabfa028e7d57ce79b49918840e59ebc990432790b66a1e86665570e942b5b0 WatchSource:0}: Error finding container 3aabfa028e7d57ce79b49918840e59ebc990432790b66a1e86665570e942b5b0: Status 404 returned error can't find the container with id 3aabfa028e7d57ce79b49918840e59ebc990432790b66a1e86665570e942b5b0
Sep 08 16:02:13.277784 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.277751    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn" event={"ID":"26e692b4-834e-4fe6-9b3d-e5976560d958","Type":"ContainerStarted","Data":"cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b"}
Sep 08 16:02:13.277784 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.277791    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn" event={"ID":"26e692b4-834e-4fe6-9b3d-e5976560d958","Type":"ContainerStarted","Data":"3aabfa028e7d57ce79b49918840e59ebc990432790b66a1e86665570e942b5b0"}
Sep 08 16:02:13.294712 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.294659    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn" podStartSLOduration=2.294642305 podStartE2EDuration="2.294642305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:02:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:02:13.292623149 +0000 UTC m=+1042.789031117" watchObservedRunningTime="2026-09-08 16:02:13.294642305 +0000 UTC m=+1042.791050272"
Sep 08 16:02:13.720164 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.720125    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:13.722559 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.722542    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:13.724510 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.724488    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5nm68\""
Sep 08 16:02:13.728423 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.728400    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:13.861564 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.861532    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvcqr\" (UniqueName: \"kubernetes.io/projected/77a2976c-411f-456d-940b-97badb9242eb-kube-api-access-cvcqr\") pod \"test-curl-7947-139872176494464\" (UID: \"77a2976c-411f-456d-940b-97badb9242eb\") " pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:13.962517 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.962479    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvcqr\" (UniqueName: \"kubernetes.io/projected/77a2976c-411f-456d-940b-97badb9242eb-kube-api-access-cvcqr\") pod \"test-curl-7947-139872176494464\" (UID: \"77a2976c-411f-456d-940b-97badb9242eb\") " pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:13.969425 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:13.969395    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvcqr\" (UniqueName: \"kubernetes.io/projected/77a2976c-411f-456d-940b-97badb9242eb-kube-api-access-cvcqr\") pod \"test-curl-7947-139872176494464\" (UID: \"77a2976c-411f-456d-940b-97badb9242eb\") " pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:14.033985 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:14.033902    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:14.302934 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:14.302908    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:14.304959 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:02:14.304934    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77a2976c_411f_456d_940b_97badb9242eb.slice/crio-35ddfd01b28c48dfd834df1fb7d54005d14b1c7f96f051d9d8b149c8a04b97b7 WatchSource:0}: Error finding container 35ddfd01b28c48dfd834df1fb7d54005d14b1c7f96f051d9d8b149c8a04b97b7: Status 404 returned error can't find the container with id 35ddfd01b28c48dfd834df1fb7d54005d14b1c7f96f051d9d8b149c8a04b97b7
Sep 08 16:02:14.306749 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:14.306732    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 16:02:15.290635 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:15.290608    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7947-139872176494464" event={"ID":"77a2976c-411f-456d-940b-97badb9242eb","Type":"ContainerStarted","Data":"0ef0f9075e795db8021673e6ec9adc29cf35320a39f9ee699b881749cca9d037"}
Sep 08 16:02:15.290732 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:15.290648    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7947-139872176494464" event={"ID":"77a2976c-411f-456d-940b-97badb9242eb","Type":"ContainerStarted","Data":"35ddfd01b28c48dfd834df1fb7d54005d14b1c7f96f051d9d8b149c8a04b97b7"}
Sep 08 16:02:16.297571 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.297538    2552 generic.go:356] "Generic (PLEG): container finished" podID="77a2976c-411f-456d-940b-97badb9242eb" containerID="0ef0f9075e795db8021673e6ec9adc29cf35320a39f9ee699b881749cca9d037" exitCode=0
Sep 08 16:02:16.297972 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.297619    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7947-139872176494464" event={"ID":"77a2976c-411f-456d-940b-97badb9242eb","Type":"ContainerDied","Data":"0ef0f9075e795db8021673e6ec9adc29cf35320a39f9ee699b881749cca9d037"}
Sep 08 16:02:16.433745 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.433719    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:16.469779 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.469749    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7947-139872176494464_77a2976c-411f-456d-940b-97badb9242eb/test-curl-7947-139872176494464/0.log"
Sep 08 16:02:16.497693 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.497649    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:16.499703 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.499683    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:16.588191 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.588105    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvcqr\" (UniqueName: \"kubernetes.io/projected/77a2976c-411f-456d-940b-97badb9242eb-kube-api-access-cvcqr\") pod \"77a2976c-411f-456d-940b-97badb9242eb\" (UID: \"77a2976c-411f-456d-940b-97badb9242eb\") "
Sep 08 16:02:16.590348 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.590311    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a2976c-411f-456d-940b-97badb9242eb-kube-api-access-cvcqr" (OuterVolumeSpecName: "kube-api-access-cvcqr") pod "77a2976c-411f-456d-940b-97badb9242eb" (UID: "77a2976c-411f-456d-940b-97badb9242eb"). InnerVolumeSpecName "kube-api-access-cvcqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:16.621895 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.621870    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:16.622307 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.622294    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a2976c-411f-456d-940b-97badb9242eb" containerName="test-curl-7947-139872176494464"
Sep 08 16:02:16.622360 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.622309    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a2976c-411f-456d-940b-97badb9242eb" containerName="test-curl-7947-139872176494464"
Sep 08 16:02:16.622396 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.622380    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a2976c-411f-456d-940b-97badb9242eb" containerName="test-curl-7947-139872176494464"
Sep 08 16:02:16.624234 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.624220    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:16.632911 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.632886    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:16.689710 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.689676    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvcqr\" (UniqueName: \"kubernetes.io/projected/77a2976c-411f-456d-940b-97badb9242eb-kube-api-access-cvcqr\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:02:16.790846 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.790812    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m8gf\" (UniqueName: \"kubernetes.io/projected/c52e4c41-5ef0-4669-a9c6-bdbad4744225-kube-api-access-2m8gf\") pod \"test-curl-7947-139872176494464\" (UID: \"c52e4c41-5ef0-4669-a9c6-bdbad4744225\") " pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:16.892145 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.892106    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m8gf\" (UniqueName: \"kubernetes.io/projected/c52e4c41-5ef0-4669-a9c6-bdbad4744225-kube-api-access-2m8gf\") pod \"test-curl-7947-139872176494464\" (UID: \"c52e4c41-5ef0-4669-a9c6-bdbad4744225\") " pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:16.900275 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.900244    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m8gf\" (UniqueName: \"kubernetes.io/projected/c52e4c41-5ef0-4669-a9c6-bdbad4744225-kube-api-access-2m8gf\") pod \"test-curl-7947-139872176494464\" (UID: \"c52e4c41-5ef0-4669-a9c6-bdbad4744225\") " pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:16.935325 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:16.935290    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:17.101294 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:17.101261    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a2976c-411f-456d-940b-97badb9242eb" path="/var/lib/kubelet/pods/77a2976c-411f-456d-940b-97badb9242eb/volumes"
Sep 08 16:02:17.219153 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:17.219019    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:17.223382 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:02:17.223342    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc52e4c41_5ef0_4669_a9c6_bdbad4744225.slice/crio-18ae8bee8249930eedd3070845b093d297e73e8a2d13aabfe461060be5595ec6 WatchSource:0}: Error finding container 18ae8bee8249930eedd3070845b093d297e73e8a2d13aabfe461060be5595ec6: Status 404 returned error can't find the container with id 18ae8bee8249930eedd3070845b093d297e73e8a2d13aabfe461060be5595ec6
Sep 08 16:02:17.304492 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:17.304466    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:17.304492 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:17.304490    2552 scope.go:117] "RemoveContainer" containerID="0ef0f9075e795db8021673e6ec9adc29cf35320a39f9ee699b881749cca9d037"
Sep 08 16:02:17.305907 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:17.305884    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7947-139872176494464" event={"ID":"c52e4c41-5ef0-4669-a9c6-bdbad4744225","Type":"ContainerStarted","Data":"18ae8bee8249930eedd3070845b093d297e73e8a2d13aabfe461060be5595ec6"}
Sep 08 16:02:20.325430 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:20.325383    2552 generic.go:356] "Generic (PLEG): container finished" podID="c52e4c41-5ef0-4669-a9c6-bdbad4744225" containerID="f157045bf31a510c36a861e9510e44e98a54cc8ae5b89594596f504b330f003d" exitCode=0
Sep 08 16:02:20.325430 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:20.325412    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7947-139872176494464" event={"ID":"c52e4c41-5ef0-4669-a9c6-bdbad4744225","Type":"ContainerDied","Data":"f157045bf31a510c36a861e9510e44e98a54cc8ae5b89594596f504b330f003d"}
Sep 08 16:02:21.462643 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:21.462618    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:21.484108 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:21.484081    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7947-139872176494464_c52e4c41-5ef0-4669-a9c6-bdbad4744225/test-curl-7947-139872176494464/0.log"
Sep 08 16:02:21.505792 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:21.505763    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:21.509423 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:21.509397    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7947-139872176494464"]
Sep 08 16:02:21.638853 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:21.638819    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m8gf\" (UniqueName: \"kubernetes.io/projected/c52e4c41-5ef0-4669-a9c6-bdbad4744225-kube-api-access-2m8gf\") pod \"c52e4c41-5ef0-4669-a9c6-bdbad4744225\" (UID: \"c52e4c41-5ef0-4669-a9c6-bdbad4744225\") "
Sep 08 16:02:21.640853 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:21.640828    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52e4c41-5ef0-4669-a9c6-bdbad4744225-kube-api-access-2m8gf" (OuterVolumeSpecName: "kube-api-access-2m8gf") pod "c52e4c41-5ef0-4669-a9c6-bdbad4744225" (UID: "c52e4c41-5ef0-4669-a9c6-bdbad4744225"). InnerVolumeSpecName "kube-api-access-2m8gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:21.740388 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:21.740355    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m8gf\" (UniqueName: \"kubernetes.io/projected/c52e4c41-5ef0-4669-a9c6-bdbad4744225-kube-api-access-2m8gf\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:02:22.084553 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:22.084466    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:22.084753 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:22.084732    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:22.097507 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:22.097479    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:22.338061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:22.337950    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:22.338061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:22.338002    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18ae8bee8249930eedd3070845b093d297e73e8a2d13aabfe461060be5595ec6"
Sep 08 16:02:22.347942 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:02:22.347915    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7947-139872176494464\" is forbidden: User \"system:node:ip-10-0-136-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-224.ec2.internal' and this object" podUID="c52e4c41-5ef0-4669-a9c6-bdbad4744225" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:22.349928 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:22.349906    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:02:22.368520 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:02:22.368485    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7947-139872176494464\" is forbidden: User \"system:node:ip-10-0-136-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-224.ec2.internal' and this object" podUID="c52e4c41-5ef0-4669-a9c6-bdbad4744225" pod="opendatahub/test-curl-7947-139872176494464"
Sep 08 16:02:23.102333 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:23.102299    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c52e4c41-5ef0-4669-a9c6-bdbad4744225" path="/var/lib/kubelet/pods/c52e4c41-5ef0-4669-a9c6-bdbad4744225/volumes"
Sep 08 16:02:33.256406 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.256372    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"]
Sep 08 16:02:33.256797 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.256784    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52e4c41-5ef0-4669-a9c6-bdbad4744225" containerName="test-curl-7947-139872176494464"
Sep 08 16:02:33.256842 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.256798    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52e4c41-5ef0-4669-a9c6-bdbad4744225" containerName="test-curl-7947-139872176494464"
Sep 08 16:02:33.256881 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.256862    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52e4c41-5ef0-4669-a9c6-bdbad4744225" containerName="test-curl-7947-139872176494464"
Sep 08 16:02:33.260260 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.260240    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.262675 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.262651    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-510a51a5\""
Sep 08 16:02:33.302575 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.302546    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"]
Sep 08 16:02:33.305988 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.305966    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.309148 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.309128    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-510a51a5-dockercfg-j5tfb\""
Sep 08 16:02:33.346404 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.346377    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"]
Sep 08 16:02:33.351813 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.351784    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-plugins-config-volume\") pod \"payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") " pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.351953 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.351841    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a826ef-6888-41ec-a802-31bc80e552a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.352043 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.352005    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzz4\" (UniqueName: \"kubernetes.io/projected/f8a826ef-6888-41ec-a802-31bc80e552a3-kube-api-access-dqzz4\") pod \"payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.352171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.352152    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8vz\" (UniqueName: \"kubernetes.io/projected/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-kube-api-access-xw8vz\") pod \"payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") " pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.376421 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.376391    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"]
Sep 08 16:02:33.453630 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.453594    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8vz\" (UniqueName: \"kubernetes.io/projected/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-kube-api-access-xw8vz\") pod \"payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") " pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.453790 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.453641    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-plugins-config-volume\") pod \"payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") " pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.453790 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.453759    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a826ef-6888-41ec-a802-31bc80e552a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.453891 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.453870    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzz4\" (UniqueName: \"kubernetes.io/projected/f8a826ef-6888-41ec-a802-31bc80e552a3-kube-api-access-dqzz4\") pod \"payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.454524 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.454489    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-plugins-config-volume\") pod \"payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") " pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.454752 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.454736    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a826ef-6888-41ec-a802-31bc80e552a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.461952 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.461926    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzz4\" (UniqueName: \"kubernetes.io/projected/f8a826ef-6888-41ec-a802-31bc80e552a3-kube-api-access-dqzz4\") pod \"payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.462088 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.461979    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8vz\" (UniqueName: \"kubernetes.io/projected/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-kube-api-access-xw8vz\") pod \"payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") " pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.570876 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.570786    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:33.616093 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.616061    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:33.848960 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.848934    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"]
Sep 08 16:02:33.850759 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:02:33.850732    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc093b3e_f849_4f9d_85f8_be5aacd9a31f.slice/crio-23f9b2e80255971a9202cacb8aa8e55255955be2f7bfab50ab3cf4efcd198401 WatchSource:0}: Error finding container 23f9b2e80255971a9202cacb8aa8e55255955be2f7bfab50ab3cf4efcd198401: Status 404 returned error can't find the container with id 23f9b2e80255971a9202cacb8aa8e55255955be2f7bfab50ab3cf4efcd198401
Sep 08 16:02:33.910529 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:33.910504    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"]
Sep 08 16:02:33.913651 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:02:33.913626    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a826ef_6888_41ec_a802_31bc80e552a3.slice/crio-db17694da1485b4993da19db775c5a00b398658fd6e2bc5b94659c1ab8e140f8 WatchSource:0}: Error finding container db17694da1485b4993da19db775c5a00b398658fd6e2bc5b94659c1ab8e140f8: Status 404 returned error can't find the container with id db17694da1485b4993da19db775c5a00b398658fd6e2bc5b94659c1ab8e140f8
Sep 08 16:02:34.417435 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.410669    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk" event={"ID":"f8a826ef-6888-41ec-a802-31bc80e552a3","Type":"ContainerStarted","Data":"94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb"}
Sep 08 16:02:34.417435 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.410782    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk" event={"ID":"f8a826ef-6888-41ec-a802-31bc80e552a3","Type":"ContainerStarted","Data":"db17694da1485b4993da19db775c5a00b398658fd6e2bc5b94659c1ab8e140f8"}
Sep 08 16:02:34.417435 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.411825    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:34.419503 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.419476    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s" event={"ID":"fc093b3e-f849-4f9d-85f8-be5aacd9a31f","Type":"ContainerStarted","Data":"830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0"}
Sep 08 16:02:34.419633 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.419531    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:34.419633 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.419553    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s" event={"ID":"fc093b3e-f849-4f9d-85f8-be5aacd9a31f","Type":"ContainerStarted","Data":"23f9b2e80255971a9202cacb8aa8e55255955be2f7bfab50ab3cf4efcd198401"}
Sep 08 16:02:34.432608 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.432558    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk" podStartSLOduration=1.4325409150000001 podStartE2EDuration="1.432540915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:02:34.431834602 +0000 UTC m=+1063.928242570" watchObservedRunningTime="2026-09-08 16:02:34.432540915 +0000 UTC m=+1063.928948877"
Sep 08 16:02:34.446642 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:34.446598    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s" podStartSLOduration=1.446579648 podStartE2EDuration="1.446579648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:02:34.446292399 +0000 UTC m=+1063.942700367" watchObservedRunningTime="2026-09-08 16:02:34.446579648 +0000 UTC m=+1063.942987619"
Sep 08 16:02:45.427518 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:45.427439    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:02:46.434079 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:46.434051    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:02:51.009889 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:02:51.009856    2552 scope.go:117] "RemoveContainer" containerID="1be28e63fa0cc90524ecfe76196bc171b03392023cadde2f7c2ccbc1cd5c2caa"
Sep 08 16:03:17.269646 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.269610    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"]
Sep 08 16:03:17.274481 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.274458    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.281038 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.280997    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-510a51a5\"/\"kube-root-ca.crt\""
Sep 08 16:03:17.281165 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.281144    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-510a51a5\"/\"e2e-lifecycle-model-510a51a5-kserve-self-signed-certs\""
Sep 08 16:03:17.281247 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.281222    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-510a51a5\"/\"default-dockercfg-bn558\""
Sep 08 16:03:17.281700 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.281010    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-510a51a5\"/\"openshift-service-ca.crt\""
Sep 08 16:03:17.289209 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.289184    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"]
Sep 08 16:03:17.395538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.395501    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tmp-dir\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.395538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.395540    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-dshm\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.395778 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.395582    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9hk7\" (UniqueName: \"kubernetes.io/projected/f75904cc-71e8-4b08-b257-84cf5dbea2a0-kube-api-access-v9hk7\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.395778 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.395617    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-home\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.395778 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.395693    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tls-certs\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.395778 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.395751    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-model-cache\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497202 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497162    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tmp-dir\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497202 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497204    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-dshm\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497455 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9hk7\" (UniqueName: \"kubernetes.io/projected/f75904cc-71e8-4b08-b257-84cf5dbea2a0-kube-api-access-v9hk7\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497455 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497291    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-home\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497455 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497352    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tls-certs\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497455 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497391    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-model-cache\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497655 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497637    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tmp-dir\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497708 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497689    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-home\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.497760 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.497740    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-model-cache\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.499468 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.499444    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-dshm\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.499805 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.499786    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tls-certs\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.505632 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.505611    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9hk7\" (UniqueName: \"kubernetes.io/projected/f75904cc-71e8-4b08-b257-84cf5dbea2a0-kube-api-access-v9hk7\") pod \"e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") " pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.584681 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.584596    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:17.863042 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:17.862997    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"]
Sep 08 16:03:17.864879 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:03:17.864852    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf75904cc_71e8_4b08_b257_84cf5dbea2a0.slice/crio-4f36dac4c81d09d3587b10addebd028086b6bb860a21960164cb9be9407c4671 WatchSource:0}: Error finding container 4f36dac4c81d09d3587b10addebd028086b6bb860a21960164cb9be9407c4671: Status 404 returned error can't find the container with id 4f36dac4c81d09d3587b10addebd028086b6bb860a21960164cb9be9407c4671
Sep 08 16:03:18.673719 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:18.673682    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb" event={"ID":"f75904cc-71e8-4b08-b257-84cf5dbea2a0","Type":"ContainerStarted","Data":"1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f"}
Sep 08 16:03:18.673719 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:18.673724    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb" event={"ID":"f75904cc-71e8-4b08-b257-84cf5dbea2a0","Type":"ContainerStarted","Data":"4f36dac4c81d09d3587b10addebd028086b6bb860a21960164cb9be9407c4671"}
Sep 08 16:03:18.690483 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:18.690437    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb" podStartSLOduration=1.690421435 podStartE2EDuration="1.690421435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:03:18.688254305 +0000 UTC m=+1108.184662275" watchObservedRunningTime="2026-09-08 16:03:18.690421435 +0000 UTC m=+1108.186829438"
Sep 08 16:03:27.585554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:27.585507    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:27.585928 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:27.585573    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:27.598562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:27.598534    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:27.734753 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:27.734722    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:03:41.812059 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.812001    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"]
Sep 08 16:03:41.816016 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.815996    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"
Sep 08 16:03:41.818035 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.817997    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:03:41.818146 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.818048    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:03:41.824889 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.824295    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"]
Sep 08 16:03:41.843999 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.843961    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac1de4-1695-418e-b6ec-de974c38bc71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-510a51a5-b49x8\" (UID: \"a4ac1de4-1695-418e-b6ec-de974c38bc71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"
Sep 08 16:03:41.945571 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.945523    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac1de4-1695-418e-b6ec-de974c38bc71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-510a51a5-b49x8\" (UID: \"a4ac1de4-1695-418e-b6ec-de974c38bc71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"
Sep 08 16:03:41.946479 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:41.946453    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac1de4-1695-418e-b6ec-de974c38bc71-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-510a51a5-b49x8\" (UID: \"a4ac1de4-1695-418e-b6ec-de974c38bc71\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"
Sep 08 16:03:42.129224 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:42.129193    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"
Sep 08 16:03:42.403789 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:42.403762    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"]
Sep 08 16:03:42.405950 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:03:42.405920    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ac1de4_1695_418e_b6ec_de974c38bc71.slice/crio-abd7d6f13dac3fbdecadf79de8d33ef300486e190454406f74d237fb19b39e08 WatchSource:0}: Error finding container abd7d6f13dac3fbdecadf79de8d33ef300486e190454406f74d237fb19b39e08: Status 404 returned error can't find the container with id abd7d6f13dac3fbdecadf79de8d33ef300486e190454406f74d237fb19b39e08
Sep 08 16:03:42.808733 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:42.808639    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8" event={"ID":"a4ac1de4-1695-418e-b6ec-de974c38bc71","Type":"ContainerStarted","Data":"9a01acdaf30999accfa52c30f467e3a430268895fdd5c61cbdc093677f123b68"}
Sep 08 16:03:42.808733 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:42.808701    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8" event={"ID":"a4ac1de4-1695-418e-b6ec-de974c38bc71","Type":"ContainerStarted","Data":"abd7d6f13dac3fbdecadf79de8d33ef300486e190454406f74d237fb19b39e08"}
Sep 08 16:03:42.826070 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:42.825999    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8" podStartSLOduration=1.825979438 podStartE2EDuration="1.825979438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:03:42.822574196 +0000 UTC m=+1132.318982164" watchObservedRunningTime="2026-09-08 16:03:42.825979438 +0000 UTC m=+1132.322387407"
Sep 08 16:03:43.816142 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:43.816106    2552 generic.go:356] "Generic (PLEG): container finished" podID="a4ac1de4-1695-418e-b6ec-de974c38bc71" containerID="9a01acdaf30999accfa52c30f467e3a430268895fdd5c61cbdc093677f123b68" exitCode=0
Sep 08 16:03:43.816341 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:43.816195    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8" event={"ID":"a4ac1de4-1695-418e-b6ec-de974c38bc71","Type":"ContainerDied","Data":"9a01acdaf30999accfa52c30f467e3a430268895fdd5c61cbdc093677f123b68"}
Sep 08 16:03:44.957196 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:44.957172    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"
Sep 08 16:03:44.974056 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:44.974010    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac1de4-1695-418e-b6ec-de974c38bc71-maas-api-service-ca\") pod \"a4ac1de4-1695-418e-b6ec-de974c38bc71\" (UID: \"a4ac1de4-1695-418e-b6ec-de974c38bc71\") "
Sep 08 16:03:44.974423 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:44.974389    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ac1de4-1695-418e-b6ec-de974c38bc71-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4ac1de4-1695-418e-b6ec-de974c38bc71" (UID: "a4ac1de4-1695-418e-b6ec-de974c38bc71"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:03:45.075133 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:45.075015    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4ac1de4-1695-418e-b6ec-de974c38bc71-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:03:45.831538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:45.831496    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8" event={"ID":"a4ac1de4-1695-418e-b6ec-de974c38bc71","Type":"ContainerDied","Data":"abd7d6f13dac3fbdecadf79de8d33ef300486e190454406f74d237fb19b39e08"}
Sep 08 16:03:45.831538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:45.831517    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"
Sep 08 16:03:45.831738 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:45.831553    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abd7d6f13dac3fbdecadf79de8d33ef300486e190454406f74d237fb19b39e08"
Sep 08 16:03:51.038239 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:51.038202    2552 scope.go:117] "RemoveContainer" containerID="f110967e448215a4d2a59ee1710b28f8375ac60405c1bb735904d3d8f7d00350"
Sep 08 16:03:51.046924 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:51.046903    2552 scope.go:117] "RemoveContainer" containerID="54defc319c367d43390c0dd098e45c32bbf01965fefdb72be5c291dd4b1e1f68"
Sep 08 16:03:51.054553 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:51.054533    2552 scope.go:117] "RemoveContainer" containerID="4f79e94f8b574575e44ebdae867e57bf3a89d5da74ebd47a77a3d8a5e367143f"
Sep 08 16:03:57.326873 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.326841    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"]
Sep 08 16:03:57.327334 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.327104    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s" podUID="fc093b3e-f849-4f9d-85f8-be5aacd9a31f" containerName="payload-processing" containerID="cri-o://830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0" gracePeriod=30
Sep 08 16:03:57.334242 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.334169    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"]
Sep 08 16:03:57.478870 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.478834    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"]
Sep 08 16:03:57.480580 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.480557    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-510a51a5-b49x8"]
Sep 08 16:03:57.586885 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.586816    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:03:57.699441 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.699406    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-plugins-config-volume\") pod \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") "
Sep 08 16:03:57.699609 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.699461    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw8vz\" (UniqueName: \"kubernetes.io/projected/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-kube-api-access-xw8vz\") pod \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\" (UID: \"fc093b3e-f849-4f9d-85f8-be5aacd9a31f\") "
Sep 08 16:03:57.699856 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.699830    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc093b3e-f849-4f9d-85f8-be5aacd9a31f" (UID: "fc093b3e-f849-4f9d-85f8-be5aacd9a31f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:03:57.701633 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.701614    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-kube-api-access-xw8vz" (OuterVolumeSpecName: "kube-api-access-xw8vz") pod "fc093b3e-f849-4f9d-85f8-be5aacd9a31f" (UID: "fc093b3e-f849-4f9d-85f8-be5aacd9a31f"). InnerVolumeSpecName "kube-api-access-xw8vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:03:57.800226 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.800194    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:03:57.800226 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.800221    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw8vz\" (UniqueName: \"kubernetes.io/projected/fc093b3e-f849-4f9d-85f8-be5aacd9a31f-kube-api-access-xw8vz\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:03:57.899682 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.899646    2552 generic.go:356] "Generic (PLEG): container finished" podID="fc093b3e-f849-4f9d-85f8-be5aacd9a31f" containerID="830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0" exitCode=0
Sep 08 16:03:57.899856 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.899691    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"
Sep 08 16:03:57.899856 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.899736    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s" event={"ID":"fc093b3e-f849-4f9d-85f8-be5aacd9a31f","Type":"ContainerDied","Data":"830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0"}
Sep 08 16:03:57.899856 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.899785    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s" event={"ID":"fc093b3e-f849-4f9d-85f8-be5aacd9a31f","Type":"ContainerDied","Data":"23f9b2e80255971a9202cacb8aa8e55255955be2f7bfab50ab3cf4efcd198401"}
Sep 08 16:03:57.899856 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.899807    2552 scope.go:117] "RemoveContainer" containerID="830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0"
Sep 08 16:03:57.900095 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.900071    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk" podUID="f8a826ef-6888-41ec-a802-31bc80e552a3" containerName="payload-pre-processing" containerID="cri-o://94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb" gracePeriod=30
Sep 08 16:03:57.910246 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.910229    2552 scope.go:117] "RemoveContainer" containerID="830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0"
Sep 08 16:03:57.910496 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:03:57.910475    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0\": container with ID starting with 830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0 not found: ID does not exist" containerID="830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0"
Sep 08 16:03:57.910564 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.910508    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0"} err="failed to get container status \"830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0\": rpc error: code = NotFound desc = could not find container \"830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0\": container with ID starting with 830f7b608968833fd51b545570f5c5b06d58ad4d2009fa27b660c0593bd9dde0 not found: ID does not exist"
Sep 08 16:03:57.937679 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.937657    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"]
Sep 08 16:03:57.945747 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:57.945724    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-510a51a5-5f5779648f-fw22s"]
Sep 08 16:03:58.139392 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.139368    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:03:58.203635 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.203547    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqzz4\" (UniqueName: \"kubernetes.io/projected/f8a826ef-6888-41ec-a802-31bc80e552a3-kube-api-access-dqzz4\") pod \"f8a826ef-6888-41ec-a802-31bc80e552a3\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") "
Sep 08 16:03:58.203635 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.203604    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a826ef-6888-41ec-a802-31bc80e552a3-plugins-config-volume\") pod \"f8a826ef-6888-41ec-a802-31bc80e552a3\" (UID: \"f8a826ef-6888-41ec-a802-31bc80e552a3\") "
Sep 08 16:03:58.203993 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.203968    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a826ef-6888-41ec-a802-31bc80e552a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8a826ef-6888-41ec-a802-31bc80e552a3" (UID: "f8a826ef-6888-41ec-a802-31bc80e552a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:03:58.205654 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.205630    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a826ef-6888-41ec-a802-31bc80e552a3-kube-api-access-dqzz4" (OuterVolumeSpecName: "kube-api-access-dqzz4") pod "f8a826ef-6888-41ec-a802-31bc80e552a3" (UID: "f8a826ef-6888-41ec-a802-31bc80e552a3"). InnerVolumeSpecName "kube-api-access-dqzz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:03:58.305111 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.305084    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqzz4\" (UniqueName: \"kubernetes.io/projected/f8a826ef-6888-41ec-a802-31bc80e552a3-kube-api-access-dqzz4\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:03:58.305111 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.305108    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8a826ef-6888-41ec-a802-31bc80e552a3-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:03:58.906160 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.906123    2552 generic.go:356] "Generic (PLEG): container finished" podID="f8a826ef-6888-41ec-a802-31bc80e552a3" containerID="94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb" exitCode=0
Sep 08 16:03:58.906627 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.906183    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"
Sep 08 16:03:58.906627 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.906209    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk" event={"ID":"f8a826ef-6888-41ec-a802-31bc80e552a3","Type":"ContainerDied","Data":"94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb"}
Sep 08 16:03:58.906627 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.906266    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk" event={"ID":"f8a826ef-6888-41ec-a802-31bc80e552a3","Type":"ContainerDied","Data":"db17694da1485b4993da19db775c5a00b398658fd6e2bc5b94659c1ab8e140f8"}
Sep 08 16:03:58.906627 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.906293    2552 scope.go:117] "RemoveContainer" containerID="94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb"
Sep 08 16:03:58.914845 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.914828    2552 scope.go:117] "RemoveContainer" containerID="94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb"
Sep 08 16:03:58.915105 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:03:58.915088    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb\": container with ID starting with 94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb not found: ID does not exist" containerID="94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb"
Sep 08 16:03:58.915161 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.915114    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb"} err="failed to get container status \"94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb\": rpc error: code = NotFound desc = could not find container \"94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb\": container with ID starting with 94da187f2f41066b72a429dedf15cec222b25197b1c6e769a115caae42d044bb not found: ID does not exist"
Sep 08 16:03:58.927998 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.927948    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"]
Sep 08 16:03:58.929929 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:58.929907    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-510a51a5-794cf8bf5d-xl7kk"]
Sep 08 16:03:59.101768 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:59.101727    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ac1de4-1695-418e-b6ec-de974c38bc71" path="/var/lib/kubelet/pods/a4ac1de4-1695-418e-b6ec-de974c38bc71/volumes"
Sep 08 16:03:59.102239 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:59.102218    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a826ef-6888-41ec-a802-31bc80e552a3" path="/var/lib/kubelet/pods/f8a826ef-6888-41ec-a802-31bc80e552a3/volumes"
Sep 08 16:03:59.102630 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:03:59.102612    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc093b3e-f849-4f9d-85f8-be5aacd9a31f" path="/var/lib/kubelet/pods/fc093b3e-f849-4f9d-85f8-be5aacd9a31f/volumes"
Sep 08 16:04:05.728584 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:05.728544    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"]
Sep 08 16:04:20.750059 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:20.749939    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb" podUID="f75904cc-71e8-4b08-b257-84cf5dbea2a0" containerName="main" containerID="cri-o://1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f" gracePeriod=45
Sep 08 16:04:20.999441 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:20.999417    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:04:21.036820 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036724    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tls-certs\") pod \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") "
Sep 08 16:04:21.036820 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036761    2552 generic.go:356] "Generic (PLEG): container finished" podID="f75904cc-71e8-4b08-b257-84cf5dbea2a0" containerID="1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f" exitCode=0
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036834    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-dshm\") pod \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") "
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036853    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036884    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb" event={"ID":"f75904cc-71e8-4b08-b257-84cf5dbea2a0","Type":"ContainerDied","Data":"1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f"}
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036921    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb" event={"ID":"f75904cc-71e8-4b08-b257-84cf5dbea2a0","Type":"ContainerDied","Data":"4f36dac4c81d09d3587b10addebd028086b6bb860a21960164cb9be9407c4671"}
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036932    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-home\") pod \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") "
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036946    2552 scope.go:117] "RemoveContainer" containerID="1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f"
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.036968    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tmp-dir\") pod \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") "
Sep 08 16:04:21.037068 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.037020    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9hk7\" (UniqueName: \"kubernetes.io/projected/f75904cc-71e8-4b08-b257-84cf5dbea2a0-kube-api-access-v9hk7\") pod \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") "
Sep 08 16:04:21.037474 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.037114    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-model-cache\") pod \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\" (UID: \"f75904cc-71e8-4b08-b257-84cf5dbea2a0\") "
Sep 08 16:04:21.037474 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.037305    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-home" (OuterVolumeSpecName: "home") pod "f75904cc-71e8-4b08-b257-84cf5dbea2a0" (UID: "f75904cc-71e8-4b08-b257-84cf5dbea2a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:04:21.037474 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.037407    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f75904cc-71e8-4b08-b257-84cf5dbea2a0" (UID: "f75904cc-71e8-4b08-b257-84cf5dbea2a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:04:21.037626 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.037544    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-home\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:04:21.037626 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.037562    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tmp-dir\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:04:21.037626 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.037566    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "f75904cc-71e8-4b08-b257-84cf5dbea2a0" (UID: "f75904cc-71e8-4b08-b257-84cf5dbea2a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:04:21.040562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.039807    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-dshm" (OuterVolumeSpecName: "dshm") pod "f75904cc-71e8-4b08-b257-84cf5dbea2a0" (UID: "f75904cc-71e8-4b08-b257-84cf5dbea2a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:04:21.040562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.039926    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f75904cc-71e8-4b08-b257-84cf5dbea2a0-kube-api-access-v9hk7" (OuterVolumeSpecName: "kube-api-access-v9hk7") pod "f75904cc-71e8-4b08-b257-84cf5dbea2a0" (UID: "f75904cc-71e8-4b08-b257-84cf5dbea2a0"). InnerVolumeSpecName "kube-api-access-v9hk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:04:21.040877 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.040848    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f75904cc-71e8-4b08-b257-84cf5dbea2a0" (UID: "f75904cc-71e8-4b08-b257-84cf5dbea2a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:04:21.060611 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.060589    2552 scope.go:117] "RemoveContainer" containerID="1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f"
Sep 08 16:04:21.060936 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:04:21.060915    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f\": container with ID starting with 1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f not found: ID does not exist" containerID="1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f"
Sep 08 16:04:21.061003 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.060951    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f"} err="failed to get container status \"1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f\": rpc error: code = NotFound desc = could not find container \"1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f\": container with ID starting with 1f1a67fc4e36332e2f71af0b60387474a9b708a60b67a29bd28fcecd5395123f not found: ID does not exist"
Sep 08 16:04:21.138121 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.138088    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-model-cache\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:04:21.138121 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.138119    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f75904cc-71e8-4b08-b257-84cf5dbea2a0-tls-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:04:21.138121 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.138128    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f75904cc-71e8-4b08-b257-84cf5dbea2a0-dshm\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:04:21.138366 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.138136    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9hk7\" (UniqueName: \"kubernetes.io/projected/f75904cc-71e8-4b08-b257-84cf5dbea2a0-kube-api-access-v9hk7\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:04:21.354245 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.354161    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"]
Sep 08 16:04:21.358137 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:21.358112    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-510a51a5/e2e-lifecycle-model-510a51a5-kserve-6b5f95cfb7-z7ghb"]
Sep 08 16:04:23.101615 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:23.101581    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f75904cc-71e8-4b08-b257-84cf5dbea2a0" path="/var/lib/kubelet/pods/f75904cc-71e8-4b08-b257-84cf5dbea2a0/volumes"
Sep 08 16:04:48.458642 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.458606    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7950-493bc9"]
Sep 08 16:04:48.459061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.458996    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a826ef-6888-41ec-a802-31bc80e552a3" containerName="payload-pre-processing"
Sep 08 16:04:48.459061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459008    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a826ef-6888-41ec-a802-31bc80e552a3" containerName="payload-pre-processing"
Sep 08 16:04:48.459061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459018    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f75904cc-71e8-4b08-b257-84cf5dbea2a0" containerName="main"
Sep 08 16:04:48.459061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459036    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f75904cc-71e8-4b08-b257-84cf5dbea2a0" containerName="main"
Sep 08 16:04:48.459061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459043    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ac1de4-1695-418e-b6ec-de974c38bc71" containerName="revoke-keys"
Sep 08 16:04:48.459061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459048    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ac1de4-1695-418e-b6ec-de974c38bc71" containerName="revoke-keys"
Sep 08 16:04:48.459061 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459065    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc093b3e-f849-4f9d-85f8-be5aacd9a31f" containerName="payload-processing"
Sep 08 16:04:48.459323 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459070    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc093b3e-f849-4f9d-85f8-be5aacd9a31f" containerName="payload-processing"
Sep 08 16:04:48.459323 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459134    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a826ef-6888-41ec-a802-31bc80e552a3" containerName="payload-pre-processing"
Sep 08 16:04:48.459323 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459145    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f75904cc-71e8-4b08-b257-84cf5dbea2a0" containerName="main"
Sep 08 16:04:48.459323 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459152    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc093b3e-f849-4f9d-85f8-be5aacd9a31f" containerName="payload-processing"
Sep 08 16:04:48.459323 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.459157    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ac1de4-1695-418e-b6ec-de974c38bc71" containerName="revoke-keys"
Sep 08 16:04:48.462033 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.462007    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:48.464044 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.464009    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5nm68\""
Sep 08 16:04:48.470179 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.469329    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7950-493bc9"]
Sep 08 16:04:48.610790 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.610750    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6qtt\" (UniqueName: \"kubernetes.io/projected/e6a7f53d-6513-4b00-8cf0-cf951b803ea2-kube-api-access-x6qtt\") pod \"mt-curl-7950-493bc9\" (UID: \"e6a7f53d-6513-4b00-8cf0-cf951b803ea2\") " pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:48.712139 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.712044    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6qtt\" (UniqueName: \"kubernetes.io/projected/e6a7f53d-6513-4b00-8cf0-cf951b803ea2-kube-api-access-x6qtt\") pod \"mt-curl-7950-493bc9\" (UID: \"e6a7f53d-6513-4b00-8cf0-cf951b803ea2\") " pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:48.720107 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.720082    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6qtt\" (UniqueName: \"kubernetes.io/projected/e6a7f53d-6513-4b00-8cf0-cf951b803ea2-kube-api-access-x6qtt\") pod \"mt-curl-7950-493bc9\" (UID: \"e6a7f53d-6513-4b00-8cf0-cf951b803ea2\") " pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:48.772521 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:48.772481    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:49.047213 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:49.047186    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7950-493bc9"]
Sep 08 16:04:49.049101 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:04:49.049065    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6a7f53d_6513_4b00_8cf0_cf951b803ea2.slice/crio-0844ca3c44e7d0cde949141f16dead40cd6ddb67519642a0548263ae733d6440 WatchSource:0}: Error finding container 0844ca3c44e7d0cde949141f16dead40cd6ddb67519642a0548263ae733d6440: Status 404 returned error can't find the container with id 0844ca3c44e7d0cde949141f16dead40cd6ddb67519642a0548263ae733d6440
Sep 08 16:04:49.196660 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:49.196628    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7950-493bc9" event={"ID":"e6a7f53d-6513-4b00-8cf0-cf951b803ea2","Type":"ContainerStarted","Data":"0844ca3c44e7d0cde949141f16dead40cd6ddb67519642a0548263ae733d6440"}
Sep 08 16:04:50.203369 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:50.203329    2552 generic.go:356] "Generic (PLEG): container finished" podID="e6a7f53d-6513-4b00-8cf0-cf951b803ea2" containerID="71c19550f6bb65f6cf7dfc009cd7b92a1d70fc32817053146b604c8f2c866390" exitCode=0
Sep 08 16:04:50.203724 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:50.203373    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7950-493bc9" event={"ID":"e6a7f53d-6513-4b00-8cf0-cf951b803ea2","Type":"ContainerDied","Data":"71c19550f6bb65f6cf7dfc009cd7b92a1d70fc32817053146b604c8f2c866390"}
Sep 08 16:04:51.085039 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.084998    2552 scope.go:117] "RemoveContainer" containerID="29af0f895282a0ec7bbf0e780a2cf8119873c52699ec0a3b1f83e388dbac0945"
Sep 08 16:04:51.347427 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.347368    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:51.376981 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.376940    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7950-493bc9_e6a7f53d-6513-4b00-8cf0-cf951b803ea2/mt-curl-7950-493bc9/0.log"
Sep 08 16:04:51.399422 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.399392    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7950-493bc9"]
Sep 08 16:04:51.401366 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.401344    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7950-493bc9"]
Sep 08 16:04:51.437357 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.437329    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6qtt\" (UniqueName: \"kubernetes.io/projected/e6a7f53d-6513-4b00-8cf0-cf951b803ea2-kube-api-access-x6qtt\") pod \"e6a7f53d-6513-4b00-8cf0-cf951b803ea2\" (UID: \"e6a7f53d-6513-4b00-8cf0-cf951b803ea2\") "
Sep 08 16:04:51.439438 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.439410    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a7f53d-6513-4b00-8cf0-cf951b803ea2-kube-api-access-x6qtt" (OuterVolumeSpecName: "kube-api-access-x6qtt") pod "e6a7f53d-6513-4b00-8cf0-cf951b803ea2" (UID: "e6a7f53d-6513-4b00-8cf0-cf951b803ea2"). InnerVolumeSpecName "kube-api-access-x6qtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:04:51.538286 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:51.538254    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6qtt\" (UniqueName: \"kubernetes.io/projected/e6a7f53d-6513-4b00-8cf0-cf951b803ea2-kube-api-access-x6qtt\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:04:52.215369 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:52.215333    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:52.215534 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:52.215391    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0844ca3c44e7d0cde949141f16dead40cd6ddb67519642a0548263ae733d6440"
Sep 08 16:04:52.224968 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:04:52.224939    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7950-493bc9\" is forbidden: User \"system:node:ip-10-0-136-224.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-224.ec2.internal' and this object" podUID="e6a7f53d-6513-4b00-8cf0-cf951b803ea2" pod="opendatahub/mt-curl-7950-493bc9"
Sep 08 16:04:53.102137 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:53.102106    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a7f53d-6513-4b00-8cf0-cf951b803ea2" path="/var/lib/kubelet/pods/e6a7f53d-6513-4b00-8cf0-cf951b803ea2/volumes"
Sep 08 16:04:57.352102 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:04:57.352061    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"]
Sep 08 16:05:06.276203 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.276165    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"]
Sep 08 16:05:06.276779 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.276759    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a7f53d-6513-4b00-8cf0-cf951b803ea2" containerName="mt-curl-7950-493bc9"
Sep 08 16:05:06.276873 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.276781    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a7f53d-6513-4b00-8cf0-cf951b803ea2" containerName="mt-curl-7950-493bc9"
Sep 08 16:05:06.276929 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.276906    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a7f53d-6513-4b00-8cf0-cf951b803ea2" containerName="mt-curl-7950-493bc9"
Sep 08 16:05:06.280142 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.280119    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.282133 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.282110    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-108124\"/\"sub-test-model-108124-kserve-self-signed-certs\""
Sep 08 16:05:06.289601 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.289577    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"]
Sep 08 16:05:06.378869 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.378835    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8kjz\" (UniqueName: \"kubernetes.io/projected/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-kube-api-access-z8kjz\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.378869 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.378878    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-model-cache\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.379110 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.378912    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-home\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.379110 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.378962    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tls-certs\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.379110 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.379034    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-dshm\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.379110 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.379075    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tmp-dir\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480240 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480201    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tls-certs\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480468 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480275    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-dshm\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480468 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480304    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tmp-dir\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480468 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480399    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8kjz\" (UniqueName: \"kubernetes.io/projected/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-kube-api-access-z8kjz\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480468 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480427    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-model-cache\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480715 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480668    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-home\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480815 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480787    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-model-cache\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480862 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480825    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tmp-dir\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.480977 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.480960    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-home\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.482594 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.482563    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-dshm\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.482884 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.482864    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tls-certs\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.487371 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.487349    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8kjz\" (UniqueName: \"kubernetes.io/projected/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-kube-api-access-z8kjz\") pod \"sub-test-model-108124-kserve-b9b5f7b7f-nrs6v\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") " pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.590311 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.590217    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:06.867419 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:06.867390    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"]
Sep 08 16:05:06.869314 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:05:06.869287    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1d2a657_cb5c_42a7_83f4_2a1fb1dcbeba.slice/crio-5d8ec1dfabb14a466365c81173fc42545791512a906c64681cef2e061906aa00 WatchSource:0}: Error finding container 5d8ec1dfabb14a466365c81173fc42545791512a906c64681cef2e061906aa00: Status 404 returned error can't find the container with id 5d8ec1dfabb14a466365c81173fc42545791512a906c64681cef2e061906aa00
Sep 08 16:05:07.300353 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:07.300275    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v" event={"ID":"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba","Type":"ContainerStarted","Data":"96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b"}
Sep 08 16:05:07.300353 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:07.300316    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v" event={"ID":"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba","Type":"ContainerStarted","Data":"5d8ec1dfabb14a466365c81173fc42545791512a906c64681cef2e061906aa00"}
Sep 08 16:05:12.371900 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.371839    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn" podUID="26e692b4-834e-4fe6-9b3d-e5976560d958" containerName="main" containerID="cri-o://cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b" gracePeriod=45
Sep 08 16:05:12.623611 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.623553    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:05:12.641252 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.641195    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v" podStartSLOduration=6.64117511 podStartE2EDuration="6.64117511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:05:07.315519469 +0000 UTC m=+1216.811927438" watchObservedRunningTime="2026-09-08 16:05:12.64117511 +0000 UTC m=+1222.137583080"
Sep 08 16:05:12.739229 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739196    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-home\") pod \"26e692b4-834e-4fe6-9b3d-e5976560d958\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") "
Sep 08 16:05:12.739393 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739288    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e692b4-834e-4fe6-9b3d-e5976560d958-tls-certs\") pod \"26e692b4-834e-4fe6-9b3d-e5976560d958\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") "
Sep 08 16:05:12.739393 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739320    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-model-cache\") pod \"26e692b4-834e-4fe6-9b3d-e5976560d958\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") "
Sep 08 16:05:12.739511 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739487    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-dshm\") pod \"26e692b4-834e-4fe6-9b3d-e5976560d958\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") "
Sep 08 16:05:12.739571 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739527    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-home" (OuterVolumeSpecName: "home") pod "26e692b4-834e-4fe6-9b3d-e5976560d958" (UID: "26e692b4-834e-4fe6-9b3d-e5976560d958"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:05:12.739571 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739531    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-tmp-dir\") pod \"26e692b4-834e-4fe6-9b3d-e5976560d958\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") "
Sep 08 16:05:12.739683 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739569    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-model-cache" (OuterVolumeSpecName: "model-cache") pod "26e692b4-834e-4fe6-9b3d-e5976560d958" (UID: "26e692b4-834e-4fe6-9b3d-e5976560d958"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:05:12.739683 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739617    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs552\" (UniqueName: \"kubernetes.io/projected/26e692b4-834e-4fe6-9b3d-e5976560d958-kube-api-access-bs552\") pod \"26e692b4-834e-4fe6-9b3d-e5976560d958\" (UID: \"26e692b4-834e-4fe6-9b3d-e5976560d958\") "
Sep 08 16:05:12.739792 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739745    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "26e692b4-834e-4fe6-9b3d-e5976560d958" (UID: "26e692b4-834e-4fe6-9b3d-e5976560d958"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:05:12.740010 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.739991    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-home\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:05:12.740010 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.740009    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-model-cache\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:05:12.740204 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.740019    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-tmp-dir\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:05:12.741516 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.741495    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e692b4-834e-4fe6-9b3d-e5976560d958-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "26e692b4-834e-4fe6-9b3d-e5976560d958" (UID: "26e692b4-834e-4fe6-9b3d-e5976560d958"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:05:12.741682 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.741652    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e692b4-834e-4fe6-9b3d-e5976560d958-kube-api-access-bs552" (OuterVolumeSpecName: "kube-api-access-bs552") pod "26e692b4-834e-4fe6-9b3d-e5976560d958" (UID: "26e692b4-834e-4fe6-9b3d-e5976560d958"). InnerVolumeSpecName "kube-api-access-bs552". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:05:12.741784 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.741703    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-dshm" (OuterVolumeSpecName: "dshm") pod "26e692b4-834e-4fe6-9b3d-e5976560d958" (UID: "26e692b4-834e-4fe6-9b3d-e5976560d958"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:05:12.841536 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.841504    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26e692b4-834e-4fe6-9b3d-e5976560d958-dshm\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:05:12.841536 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.841534    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs552\" (UniqueName: \"kubernetes.io/projected/26e692b4-834e-4fe6-9b3d-e5976560d958-kube-api-access-bs552\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:05:12.841536 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:12.841544    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26e692b4-834e-4fe6-9b3d-e5976560d958-tls-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:05:13.335179 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.335143    2552 generic.go:356] "Generic (PLEG): container finished" podID="26e692b4-834e-4fe6-9b3d-e5976560d958" containerID="cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b" exitCode=0
Sep 08 16:05:13.335179 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.335182    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"
Sep 08 16:05:13.335400 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.335228    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn" event={"ID":"26e692b4-834e-4fe6-9b3d-e5976560d958","Type":"ContainerDied","Data":"cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b"}
Sep 08 16:05:13.335400 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.335272    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn" event={"ID":"26e692b4-834e-4fe6-9b3d-e5976560d958","Type":"ContainerDied","Data":"3aabfa028e7d57ce79b49918840e59ebc990432790b66a1e86665570e942b5b0"}
Sep 08 16:05:13.335400 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.335293    2552 scope.go:117] "RemoveContainer" containerID="cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b"
Sep 08 16:05:13.343916 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.343899    2552 scope.go:117] "RemoveContainer" containerID="cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b"
Sep 08 16:05:13.344270 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:05:13.344235    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b\": container with ID starting with cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b not found: ID does not exist" containerID="cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b"
Sep 08 16:05:13.344333 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.344277    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b"} err="failed to get container status \"cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b\": rpc error: code = NotFound desc = could not find container \"cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b\": container with ID starting with cfdaaff93ead56ad99c765f0c9addc111eb8ff378c531e346499a83b8426189b not found: ID does not exist"
Sep 08 16:05:13.350190 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.350167    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"]
Sep 08 16:05:13.357683 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:13.357660    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/auth-test-model-108124-kserve-795bd74799-p8sfn"]
Sep 08 16:05:15.107768 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:15.107724    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26e692b4-834e-4fe6-9b3d-e5976560d958" path="/var/lib/kubelet/pods/26e692b4-834e-4fe6-9b3d-e5976560d958/volumes"
Sep 08 16:05:16.591427 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:16.591391    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:16.591810 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:16.591450    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:16.604466 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:16.604441    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:17.370536 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:17.370507    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:05:51.117774 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:51.117699    2552 scope.go:117] "RemoveContainer" containerID="243a36267abecee0b10025638fedc84e441596a849b975d80bd010ab12e0f4f8"
Sep 08 16:05:57.058347 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.058315    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"]
Sep 08 16:05:57.058818 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.058705    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e692b4-834e-4fe6-9b3d-e5976560d958" containerName="main"
Sep 08 16:05:57.058818 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.058717    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e692b4-834e-4fe6-9b3d-e5976560d958" containerName="main"
Sep 08 16:05:57.058818 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.058795    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e692b4-834e-4fe6-9b3d-e5976560d958" containerName="main"
Sep 08 16:05:57.066951 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.066916    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.069578 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.069549    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5214de-openshift-default-dockercfg-n92cl\""
Sep 08 16:05:57.075312 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.075287    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"]
Sep 08 16:05:57.170403 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170368    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-data\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170423    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-credential-socket\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170550    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-istio-token\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170599    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f962868-b96c-41ee-8bc0-f038095a0085-istio-podinfo\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170755 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170646    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gshm9\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-kube-api-access-gshm9\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170755 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170679    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-envoy\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170824 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170766    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f962868-b96c-41ee-8bc0-f038095a0085-istiod-ca-cert\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170824 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170800    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-certs\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.170917 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.170831    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-socket\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272143 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272103    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f962868-b96c-41ee-8bc0-f038095a0085-istiod-ca-cert\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272326 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272151    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-certs\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272326 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272176    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-socket\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272326 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272210    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-data\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272326 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-credential-socket\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272326 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272304    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-istio-token\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272581 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272341    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f962868-b96c-41ee-8bc0-f038095a0085-istio-podinfo\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272581 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272375    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gshm9\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-kube-api-access-gshm9\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272581 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272402    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-envoy\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272581 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272567    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-certs\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272766 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272600    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-data\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272766 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272654    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-credential-socket\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272766 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272705    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-socket\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.272964 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.272942    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f962868-b96c-41ee-8bc0-f038095a0085-istiod-ca-cert\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.274624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.274603    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-envoy\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.274960 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.274943    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f962868-b96c-41ee-8bc0-f038095a0085-istio-podinfo\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.279290 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.279269    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-istio-token\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.279810 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.279790    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gshm9\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-kube-api-access-gshm9\") pod \"e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") " pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.381678 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.381651    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:57.663334 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.663310    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"]
Sep 08 16:05:57.665264 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:05:57.665234    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f962868_b96c_41ee_8bc0_f038095a0085.slice/crio-075d9c4bff62b2b93dae8d8184ae303f8e650605f692584f49cb33b79f346bd2 WatchSource:0}: Error finding container 075d9c4bff62b2b93dae8d8184ae303f8e650605f692584f49cb33b79f346bd2: Status 404 returned error can't find the container with id 075d9c4bff62b2b93dae8d8184ae303f8e650605f692584f49cb33b79f346bd2
Sep 08 16:05:57.667664 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.667618    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:05:57.667761 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.667742    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:05:57.667834 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:57.667791    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:05:58.590454 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:58.590416    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" event={"ID":"4f962868-b96c-41ee-8bc0-f038095a0085","Type":"ContainerStarted","Data":"2abde71f4bd5ce7e75fcaebc1d862b2e390f4fcddab429161691c5f66d623dd1"}
Sep 08 16:05:58.590838 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:58.590476    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" event={"ID":"4f962868-b96c-41ee-8bc0-f038095a0085","Type":"ContainerStarted","Data":"075d9c4bff62b2b93dae8d8184ae303f8e650605f692584f49cb33b79f346bd2"}
Sep 08 16:05:58.611423 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:58.611376    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" podStartSLOduration=1.611362159 podStartE2EDuration="1.611362159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:05:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:05:58.609364573 +0000 UTC m=+1268.105772545" watchObservedRunningTime="2026-09-08 16:05:58.611362159 +0000 UTC m=+1268.107770180"
Sep 08 16:05:59.382373 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:59.382329    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:59.388169 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:59.388144    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:59.596396 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:59.596363    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:05:59.597437 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:05:59.597416    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:06:02.644076 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.644038    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"]
Sep 08 16:06:02.647785 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.647766    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:02.649933 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.649908    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-5214de-dockercfg-gl5rc\""
Sep 08 16:06:02.650085 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.649908    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-5214de\""
Sep 08 16:06:02.657841 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.657818    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"]
Sep 08 16:06:02.732828 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.732786    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwmgw\" (UniqueName: \"kubernetes.io/projected/e71d3d73-7eba-417f-bd22-974048ff8fe2-kube-api-access-dwmgw\") pod \"payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:02.733186 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.733153    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e71d3d73-7eba-417f-bd22-974048ff8fe2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:02.834199 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.834166    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwmgw\" (UniqueName: \"kubernetes.io/projected/e71d3d73-7eba-417f-bd22-974048ff8fe2-kube-api-access-dwmgw\") pod \"payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:02.834384 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.834220    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e71d3d73-7eba-417f-bd22-974048ff8fe2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:02.834997 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.834970    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e71d3d73-7eba-417f-bd22-974048ff8fe2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:02.842388 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.842356    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwmgw\" (UniqueName: \"kubernetes.io/projected/e71d3d73-7eba-417f-bd22-974048ff8fe2-kube-api-access-dwmgw\") pod \"payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:02.962878 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:02.962785    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:03.239506 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:03.239477    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"]
Sep 08 16:06:03.241357 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:06:03.241328    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71d3d73_7eba_417f_bd22_974048ff8fe2.slice/crio-7c5ddff7d784aa2d835009af31c76046a6a92d90bfed7f3883274cad0c15bf1c WatchSource:0}: Error finding container 7c5ddff7d784aa2d835009af31c76046a6a92d90bfed7f3883274cad0c15bf1c: Status 404 returned error can't find the container with id 7c5ddff7d784aa2d835009af31c76046a6a92d90bfed7f3883274cad0c15bf1c
Sep 08 16:06:03.620424 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:03.620384    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg" event={"ID":"e71d3d73-7eba-417f-bd22-974048ff8fe2","Type":"ContainerStarted","Data":"b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d"}
Sep 08 16:06:03.620590 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:03.620450    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg" event={"ID":"e71d3d73-7eba-417f-bd22-974048ff8fe2","Type":"ContainerStarted","Data":"7c5ddff7d784aa2d835009af31c76046a6a92d90bfed7f3883274cad0c15bf1c"}
Sep 08 16:06:03.620590 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:03.620491    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:03.639355 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:03.639298    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg" podStartSLOduration=1.639281177 podStartE2EDuration="1.639281177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:06:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:06:03.63718866 +0000 UTC m=+1273.133596630" watchObservedRunningTime="2026-09-08 16:06:03.639281177 +0000 UTC m=+1273.135689144"
Sep 08 16:06:06.973523 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:06.973486    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"]
Sep 08 16:06:14.629269 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:14.629228    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:06:21.730926 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:21.730890    2552 generic.go:356] "Generic (PLEG): container finished" podID="4f962868-b96c-41ee-8bc0-f038095a0085" containerID="2abde71f4bd5ce7e75fcaebc1d862b2e390f4fcddab429161691c5f66d623dd1" exitCode=0
Sep 08 16:06:21.731380 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:21.730961    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" event={"ID":"4f962868-b96c-41ee-8bc0-f038095a0085","Type":"ContainerDied","Data":"2abde71f4bd5ce7e75fcaebc1d862b2e390f4fcddab429161691c5f66d623dd1"}
Sep 08 16:06:21.731483 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:21.731468    2552 scope.go:117] "RemoveContainer" containerID="2abde71f4bd5ce7e75fcaebc1d862b2e390f4fcddab429161691c5f66d623dd1"
Sep 08 16:06:21.732676 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:21.732644    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:06:21.732772 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:21.732718    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:06:21.732772 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:21.732750    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:06:21.993751 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:21.993655    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v" podUID="a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" containerName="main" containerID="cri-o://96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b" gracePeriod=45
Sep 08 16:06:22.243892 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.243836    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:06:22.344983 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.344945    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tls-certs\") pod \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") "
Sep 08 16:06:22.344983 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.344988    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tmp-dir\") pod \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") "
Sep 08 16:06:22.345244 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345020    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8kjz\" (UniqueName: \"kubernetes.io/projected/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-kube-api-access-z8kjz\") pod \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") "
Sep 08 16:06:22.345244 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345137    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-home\") pod \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") "
Sep 08 16:06:22.345244 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345176    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-dshm\") pod \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") "
Sep 08 16:06:22.345244 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345240    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-model-cache\") pod \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\" (UID: \"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba\") "
Sep 08 16:06:22.345553 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345297    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" (UID: "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:06:22.345553 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345484    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-home" (OuterVolumeSpecName: "home") pod "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" (UID: "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:06:22.345661 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345640    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tmp-dir\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:06:22.345706 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345663    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-home\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:06:22.345706 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.345671    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" (UID: "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:06:22.347325 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.347302    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-kube-api-access-z8kjz" (OuterVolumeSpecName: "kube-api-access-z8kjz") pod "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" (UID: "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba"). InnerVolumeSpecName "kube-api-access-z8kjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:06:22.347435 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.347321    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" (UID: "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:06:22.347606 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.347585    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-dshm" (OuterVolumeSpecName: "dshm") pod "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" (UID: "a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:06:22.446791 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.446681    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-tls-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:06:22.446791 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.446723    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8kjz\" (UniqueName: \"kubernetes.io/projected/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-kube-api-access-z8kjz\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:06:22.446791 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.446739    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-dshm\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:06:22.446791 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.446753    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba-model-cache\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:06:22.746712 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.746666    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" event={"ID":"4f962868-b96c-41ee-8bc0-f038095a0085","Type":"ContainerStarted","Data":"87a2f073d8333c68c11b097b9d059dbb4cff13a0d1dccea8527b3fcaec17a7ae"}
Sep 08 16:06:22.749571 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.749534    2552 generic.go:356] "Generic (PLEG): container finished" podID="a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" containerID="96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b" exitCode=0
Sep 08 16:06:22.749724 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.749648    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v" event={"ID":"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba","Type":"ContainerDied","Data":"96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b"}
Sep 08 16:06:22.749724 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.749679    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v" event={"ID":"a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba","Type":"ContainerDied","Data":"5d8ec1dfabb14a466365c81173fc42545791512a906c64681cef2e061906aa00"}
Sep 08 16:06:22.749724 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.749701    2552 scope.go:117] "RemoveContainer" containerID="96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b"
Sep 08 16:06:22.749899 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.749855    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"
Sep 08 16:06:22.765809 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.765783    2552 scope.go:117] "RemoveContainer" containerID="96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b"
Sep 08 16:06:22.766200 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:06:22.766176    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b\": container with ID starting with 96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b not found: ID does not exist" containerID="96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b"
Sep 08 16:06:22.766288 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.766213    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b"} err="failed to get container status \"96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b\": rpc error: code = NotFound desc = could not find container \"96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b\": container with ID starting with 96c6350abd65c10769d4d72494e644aa371d2e1168a5be2a06390a542c011d1b not found: ID does not exist"
Sep 08 16:06:22.787614 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.787571    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"]
Sep 08 16:06:22.791206 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:22.791181    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-108124/sub-test-model-108124-kserve-b9b5f7b7f-nrs6v"]
Sep 08 16:06:23.101599 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:23.101511    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" path="/var/lib/kubelet/pods/a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba/volumes"
Sep 08 16:06:23.382594 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:23.382560    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:06:23.382812 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:23.382587    2552 patch_prober.go:28] interesting pod/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 08 16:06:23.382812 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:23.382690    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 08 16:06:24.382077 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:24.382010    2552 patch_prober.go:28] interesting pod/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 08 16:06:24.382538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:24.382104    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 08 16:06:25.382934 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:25.382894    2552 patch_prober.go:28] interesting pod/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 08 16:06:25.383367 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:25.382970    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 08 16:06:26.382650 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:26.382616    2552 patch_prober.go:28] interesting pod/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 08 16:06:26.382850 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:26.382682    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 08 16:06:27.382709 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:27.382676    2552 patch_prober.go:28] interesting pod/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 08 16:06:27.383172 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:27.382728    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 08 16:06:28.386653 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:28.386623    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:06:28.387089 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:28.386953    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:06:28.387656 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:28.387640    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:06:30.677284 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.677252    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"]
Sep 08 16:06:30.677738 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.677709    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" containerName="main"
Sep 08 16:06:30.677738 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.677721    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" containerName="main"
Sep 08 16:06:30.677865 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.677797    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1d2a657-cb5c-42a7-83f4-2a1fb1dcbeba" containerName="main"
Sep 08 16:06:30.680922 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.680900    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:30.683223 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.683198    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b8a099\""
Sep 08 16:06:30.690920 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.690894    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"]
Sep 08 16:06:30.718039 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.718003    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"]
Sep 08 16:06:30.721337 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.721318    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:30.723622 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.723597    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b8a099-dockercfg-rqwkx\""
Sep 08 16:06:30.732569 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.732536    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"]
Sep 08 16:06:30.735875 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.735844    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd4ww\" (UniqueName: \"kubernetes.io/projected/761ed289-225f-4a66-987b-33dc6ddef779-kube-api-access-zd4ww\") pod \"payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:30.736017 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.735997    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/761ed289-225f-4a66-987b-33dc6ddef779-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:30.837279 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.837242    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633caf66-6283-4fcb-8326-2aa4178059a4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") " pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:30.837279 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.837282    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/761ed289-225f-4a66-987b-33dc6ddef779-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:30.837485 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.837407    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-725r5\" (UniqueName: \"kubernetes.io/projected/633caf66-6283-4fcb-8326-2aa4178059a4-kube-api-access-725r5\") pod \"payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") " pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:30.837485 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.837445    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd4ww\" (UniqueName: \"kubernetes.io/projected/761ed289-225f-4a66-987b-33dc6ddef779-kube-api-access-zd4ww\") pod \"payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:30.837919 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.837900    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/761ed289-225f-4a66-987b-33dc6ddef779-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:30.851720 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.851684    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd4ww\" (UniqueName: \"kubernetes.io/projected/761ed289-225f-4a66-987b-33dc6ddef779-kube-api-access-zd4ww\") pod \"payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:30.939078 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.938969    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633caf66-6283-4fcb-8326-2aa4178059a4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") " pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:30.939252 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.939110    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-725r5\" (UniqueName: \"kubernetes.io/projected/633caf66-6283-4fcb-8326-2aa4178059a4-kube-api-access-725r5\") pod \"payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") " pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:30.939673 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.939651    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633caf66-6283-4fcb-8326-2aa4178059a4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") " pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:30.946909 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.946880    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-725r5\" (UniqueName: \"kubernetes.io/projected/633caf66-6283-4fcb-8326-2aa4178059a4-kube-api-access-725r5\") pod \"payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") " pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:30.994554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:30.994520    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:31.035095 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.034738    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:31.270021 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.269997    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"]
Sep 08 16:06:31.271772 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:06:31.271747    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761ed289_225f_4a66_987b_33dc6ddef779.slice/crio-e39cc05e06670b9d61a5966ea4a0b3bc88f6a8ed86d8f47446c3edd130eec526 WatchSource:0}: Error finding container e39cc05e06670b9d61a5966ea4a0b3bc88f6a8ed86d8f47446c3edd130eec526: Status 404 returned error can't find the container with id e39cc05e06670b9d61a5966ea4a0b3bc88f6a8ed86d8f47446c3edd130eec526
Sep 08 16:06:31.317270 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.317244    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"]
Sep 08 16:06:31.318967 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:06:31.318933    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633caf66_6283_4fcb_8326_2aa4178059a4.slice/crio-68ba8f15a083b0c5f1fb01b2be2089e01cdd6e1652cb02ec460793fc55f29609 WatchSource:0}: Error finding container 68ba8f15a083b0c5f1fb01b2be2089e01cdd6e1652cb02ec460793fc55f29609: Status 404 returned error can't find the container with id 68ba8f15a083b0c5f1fb01b2be2089e01cdd6e1652cb02ec460793fc55f29609
Sep 08 16:06:31.815168 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.815128    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk" event={"ID":"633caf66-6283-4fcb-8326-2aa4178059a4","Type":"ContainerStarted","Data":"c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624"}
Sep 08 16:06:31.815628 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.815187    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk" event={"ID":"633caf66-6283-4fcb-8326-2aa4178059a4","Type":"ContainerStarted","Data":"68ba8f15a083b0c5f1fb01b2be2089e01cdd6e1652cb02ec460793fc55f29609"}
Sep 08 16:06:31.815628 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.815218    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:31.816965 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.816927    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp" event={"ID":"761ed289-225f-4a66-987b-33dc6ddef779","Type":"ContainerStarted","Data":"d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c"}
Sep 08 16:06:31.817111 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.816983    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:31.817111 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.817004    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp" event={"ID":"761ed289-225f-4a66-987b-33dc6ddef779","Type":"ContainerStarted","Data":"e39cc05e06670b9d61a5966ea4a0b3bc88f6a8ed86d8f47446c3edd130eec526"}
Sep 08 16:06:31.833780 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.833705    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk" podStartSLOduration=1.8336797310000001 podStartE2EDuration="1.833679731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:06:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:06:31.83156693 +0000 UTC m=+1301.327974901" watchObservedRunningTime="2026-09-08 16:06:31.833679731 +0000 UTC m=+1301.330087699"
Sep 08 16:06:31.849515 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:31.849461    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp" podStartSLOduration=1.849440957 podStartE2EDuration="1.849440957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:06:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:06:31.848669124 +0000 UTC m=+1301.345077092" watchObservedRunningTime="2026-09-08 16:06:31.849440957 +0000 UTC m=+1301.345848926"
Sep 08 16:06:42.825174 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:42.825145    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:06:42.825778 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:42.825213    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:06:51.144394 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:51.144364    2552 scope.go:117] "RemoveContainer" containerID="017fb4c70db05d0cc58e24b1a2a7fdeedb5b6aee8cab254c79232d2135fc7e58"
Sep 08 16:06:51.156706 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:51.156686    2552 scope.go:117] "RemoveContainer" containerID="45f0a6d176255d4bce0b27864afe112bc01e4aeb910cb9d274fc4b0128d82153"
Sep 08 16:06:51.164982 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:51.164956    2552 scope.go:117] "RemoveContainer" containerID="50ab601e6c7e9923e5968dd991681659f409f23e7ce8fa09eb504d343e207225"
Sep 08 16:06:56.629634 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.629600    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"]
Sep 08 16:06:56.633456 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.633435    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"
Sep 08 16:06:56.635884 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.635858    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:06:56.636002 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.635886    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:06:56.642817 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.642792    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"]
Sep 08 16:06:56.691661 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.691626    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb883ee3-2af9-4f25-87c2-70d8d93b228a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5214de-znk8x\" (UID: \"eb883ee3-2af9-4f25-87c2-70d8d93b228a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"
Sep 08 16:06:56.792859 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.792823    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb883ee3-2af9-4f25-87c2-70d8d93b228a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5214de-znk8x\" (UID: \"eb883ee3-2af9-4f25-87c2-70d8d93b228a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"
Sep 08 16:06:56.793525 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.793497    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb883ee3-2af9-4f25-87c2-70d8d93b228a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-5214de-znk8x\" (UID: \"eb883ee3-2af9-4f25-87c2-70d8d93b228a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"
Sep 08 16:06:56.945166 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:56.945080    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"
Sep 08 16:06:57.217076 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:57.216963    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"]
Sep 08 16:06:57.220012 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:06:57.219973    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb883ee3_2af9_4f25_87c2_70d8d93b228a.slice/crio-56de1e686a3d16d78ad6625c0726726082cbe32e441cc25613af992fbf0ffbb0 WatchSource:0}: Error finding container 56de1e686a3d16d78ad6625c0726726082cbe32e441cc25613af992fbf0ffbb0: Status 404 returned error can't find the container with id 56de1e686a3d16d78ad6625c0726726082cbe32e441cc25613af992fbf0ffbb0
Sep 08 16:06:57.977383 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:57.977344    2552 generic.go:356] "Generic (PLEG): container finished" podID="eb883ee3-2af9-4f25-87c2-70d8d93b228a" containerID="934322789d31144a7e06bac3b2e970ccc36870918ed832355a7817eecdc9f611" exitCode=0
Sep 08 16:06:57.977747 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:57.977420    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x" event={"ID":"eb883ee3-2af9-4f25-87c2-70d8d93b228a","Type":"ContainerDied","Data":"934322789d31144a7e06bac3b2e970ccc36870918ed832355a7817eecdc9f611"}
Sep 08 16:06:57.977747 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:57.977472    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x" event={"ID":"eb883ee3-2af9-4f25-87c2-70d8d93b228a","Type":"ContainerStarted","Data":"56de1e686a3d16d78ad6625c0726726082cbe32e441cc25613af992fbf0ffbb0"}
Sep 08 16:06:59.123672 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:59.123647    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"
Sep 08 16:06:59.220015 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:59.219982    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb883ee3-2af9-4f25-87c2-70d8d93b228a-maas-api-service-ca\") pod \"eb883ee3-2af9-4f25-87c2-70d8d93b228a\" (UID: \"eb883ee3-2af9-4f25-87c2-70d8d93b228a\") "
Sep 08 16:06:59.220303 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:59.220282    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb883ee3-2af9-4f25-87c2-70d8d93b228a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eb883ee3-2af9-4f25-87c2-70d8d93b228a" (UID: "eb883ee3-2af9-4f25-87c2-70d8d93b228a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:06:59.321624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:59.321523    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb883ee3-2af9-4f25-87c2-70d8d93b228a-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:06:59.989994 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:59.989964    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"
Sep 08 16:06:59.990194 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:59.989963    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x" event={"ID":"eb883ee3-2af9-4f25-87c2-70d8d93b228a","Type":"ContainerDied","Data":"56de1e686a3d16d78ad6625c0726726082cbe32e441cc25613af992fbf0ffbb0"}
Sep 08 16:06:59.990194 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:06:59.990086    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56de1e686a3d16d78ad6625c0726726082cbe32e441cc25613af992fbf0ffbb0"
Sep 08 16:07:12.528237 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.528197    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"]
Sep 08 16:07:12.528788 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.528586    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg" podUID="e71d3d73-7eba-417f-bd22-974048ff8fe2" containerName="payload-processing" containerID="cri-o://b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d" gracePeriod=30
Sep 08 16:07:12.698766 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.698723    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"]
Sep 08 16:07:12.701205 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.700852    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-5214de-znk8x"]
Sep 08 16:07:12.788805 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.788361    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:07:12.854084 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.854043    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwmgw\" (UniqueName: \"kubernetes.io/projected/e71d3d73-7eba-417f-bd22-974048ff8fe2-kube-api-access-dwmgw\") pod \"e71d3d73-7eba-417f-bd22-974048ff8fe2\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") "
Sep 08 16:07:12.854309 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.854147    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e71d3d73-7eba-417f-bd22-974048ff8fe2-plugins-config-volume\") pod \"e71d3d73-7eba-417f-bd22-974048ff8fe2\" (UID: \"e71d3d73-7eba-417f-bd22-974048ff8fe2\") "
Sep 08 16:07:12.854525 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.854493    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e71d3d73-7eba-417f-bd22-974048ff8fe2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e71d3d73-7eba-417f-bd22-974048ff8fe2" (UID: "e71d3d73-7eba-417f-bd22-974048ff8fe2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:07:12.856076 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.856019    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e71d3d73-7eba-417f-bd22-974048ff8fe2-kube-api-access-dwmgw" (OuterVolumeSpecName: "kube-api-access-dwmgw") pod "e71d3d73-7eba-417f-bd22-974048ff8fe2" (UID: "e71d3d73-7eba-417f-bd22-974048ff8fe2"). InnerVolumeSpecName "kube-api-access-dwmgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:07:12.955324 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.955285    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwmgw\" (UniqueName: \"kubernetes.io/projected/e71d3d73-7eba-417f-bd22-974048ff8fe2-kube-api-access-dwmgw\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:12.955324 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:12.955315    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e71d3d73-7eba-417f-bd22-974048ff8fe2-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:13.062409 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.062313    2552 generic.go:356] "Generic (PLEG): container finished" podID="e71d3d73-7eba-417f-bd22-974048ff8fe2" containerID="b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d" exitCode=0
Sep 08 16:07:13.062409 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.062373    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"
Sep 08 16:07:13.062638 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.062405    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg" event={"ID":"e71d3d73-7eba-417f-bd22-974048ff8fe2","Type":"ContainerDied","Data":"b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d"}
Sep 08 16:07:13.062638 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.062499    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg" event={"ID":"e71d3d73-7eba-417f-bd22-974048ff8fe2","Type":"ContainerDied","Data":"7c5ddff7d784aa2d835009af31c76046a6a92d90bfed7f3883274cad0c15bf1c"}
Sep 08 16:07:13.062638 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.062522    2552 scope.go:117] "RemoveContainer" containerID="b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d"
Sep 08 16:07:13.071889 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.071872    2552 scope.go:117] "RemoveContainer" containerID="b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d"
Sep 08 16:07:13.072180 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:07:13.072162    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d\": container with ID starting with b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d not found: ID does not exist" containerID="b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d"
Sep 08 16:07:13.072233 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.072186    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d"} err="failed to get container status \"b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d\": rpc error: code = NotFound desc = could not find container \"b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d\": container with ID starting with b5819804e5d757c6921b76f60ee3ddcd840f55399fa87605fae190c713683f2d not found: ID does not exist"
Sep 08 16:07:13.083187 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.083159    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"]
Sep 08 16:07:13.089066 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.089044    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5214de-f569bf9bf-hpvvg"]
Sep 08 16:07:13.102058 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.102018    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e71d3d73-7eba-417f-bd22-974048ff8fe2" path="/var/lib/kubelet/pods/e71d3d73-7eba-417f-bd22-974048ff8fe2/volumes"
Sep 08 16:07:13.102394 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:13.102380    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb883ee3-2af9-4f25-87c2-70d8d93b228a" path="/var/lib/kubelet/pods/eb883ee3-2af9-4f25-87c2-70d8d93b228a/volumes"
Sep 08 16:07:18.995655 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:18.995619    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"]
Sep 08 16:07:18.996172 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:18.995928    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy" containerID="cri-o://87a2f073d8333c68c11b097b9d059dbb4cff13a0d1dccea8527b3fcaec17a7ae" gracePeriod=30
Sep 08 16:07:19.341546 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.341461    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"]
Sep 08 16:07:19.341926 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.341903    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e71d3d73-7eba-417f-bd22-974048ff8fe2" containerName="payload-processing"
Sep 08 16:07:19.341926 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.341926    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e71d3d73-7eba-417f-bd22-974048ff8fe2" containerName="payload-processing"
Sep 08 16:07:19.342167 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.341938    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb883ee3-2af9-4f25-87c2-70d8d93b228a" containerName="revoke-keys"
Sep 08 16:07:19.342167 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.341944    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb883ee3-2af9-4f25-87c2-70d8d93b228a" containerName="revoke-keys"
Sep 08 16:07:19.342167 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.342092    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e71d3d73-7eba-417f-bd22-974048ff8fe2" containerName="payload-processing"
Sep 08 16:07:19.342167 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.342110    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb883ee3-2af9-4f25-87c2-70d8d93b228a" containerName="revoke-keys"
Sep 08 16:07:19.345286 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.345270    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"
Sep 08 16:07:19.347431 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.347411    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:07:19.347539 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.347412    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:07:19.355010 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.354986    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"]
Sep 08 16:07:19.517081 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.517011    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/def9d97e-16c7-4170-a18e-b11b88ef544e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf\" (UID: \"def9d97e-16c7-4170-a18e-b11b88ef544e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"
Sep 08 16:07:19.617988 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.617954    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/def9d97e-16c7-4170-a18e-b11b88ef544e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf\" (UID: \"def9d97e-16c7-4170-a18e-b11b88ef544e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"
Sep 08 16:07:19.618615 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.618593    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/def9d97e-16c7-4170-a18e-b11b88ef544e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf\" (UID: \"def9d97e-16c7-4170-a18e-b11b88ef544e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"
Sep 08 16:07:19.657936 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.657900    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"
Sep 08 16:07:19.936306 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:19.936277    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"]
Sep 08 16:07:19.938545 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:07:19.938515    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef9d97e_16c7_4170_a18e_b11b88ef544e.slice/crio-4c2c36fc0a124106e24ce19d72b7321afca37721acc6e2fd580e682fc903f2ef WatchSource:0}: Error finding container 4c2c36fc0a124106e24ce19d72b7321afca37721acc6e2fd580e682fc903f2ef: Status 404 returned error can't find the container with id 4c2c36fc0a124106e24ce19d72b7321afca37721acc6e2fd580e682fc903f2ef
Sep 08 16:07:20.104585 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:20.104548    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf" event={"ID":"def9d97e-16c7-4170-a18e-b11b88ef544e","Type":"ContainerStarted","Data":"4c2c36fc0a124106e24ce19d72b7321afca37721acc6e2fd580e682fc903f2ef"}
Sep 08 16:07:21.110538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:21.110507    2552 generic.go:356] "Generic (PLEG): container finished" podID="def9d97e-16c7-4170-a18e-b11b88ef544e" containerID="f9096217da119f80df6cab420d534e8cdf65db84f516a3f2e4f862bb7fd7f056" exitCode=0
Sep 08 16:07:21.110956 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:21.110561    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf" event={"ID":"def9d97e-16c7-4170-a18e-b11b88ef544e","Type":"ContainerDied","Data":"f9096217da119f80df6cab420d534e8cdf65db84f516a3f2e4f862bb7fd7f056"}
Sep 08 16:07:22.249638 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:22.249614    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"
Sep 08 16:07:22.345877 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:22.345844    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/def9d97e-16c7-4170-a18e-b11b88ef544e-maas-api-service-ca\") pod \"def9d97e-16c7-4170-a18e-b11b88ef544e\" (UID: \"def9d97e-16c7-4170-a18e-b11b88ef544e\") "
Sep 08 16:07:22.346225 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:22.346203    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/def9d97e-16c7-4170-a18e-b11b88ef544e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "def9d97e-16c7-4170-a18e-b11b88ef544e" (UID: "def9d97e-16c7-4170-a18e-b11b88ef544e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:07:22.447599 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:22.447513    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/def9d97e-16c7-4170-a18e-b11b88ef544e-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:23.122500 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:23.122476    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"
Sep 08 16:07:23.122656 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:23.122516    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf" event={"ID":"def9d97e-16c7-4170-a18e-b11b88ef544e","Type":"ContainerDied","Data":"4c2c36fc0a124106e24ce19d72b7321afca37721acc6e2fd580e682fc903f2ef"}
Sep 08 16:07:23.122656 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:23.122560    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c2c36fc0a124106e24ce19d72b7321afca37721acc6e2fd580e682fc903f2ef"
Sep 08 16:07:24.132478 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.132449    2552 generic.go:356] "Generic (PLEG): container finished" podID="4f962868-b96c-41ee-8bc0-f038095a0085" containerID="87a2f073d8333c68c11b097b9d059dbb4cff13a0d1dccea8527b3fcaec17a7ae" exitCode=0
Sep 08 16:07:24.132842 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.132537    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" event={"ID":"4f962868-b96c-41ee-8bc0-f038095a0085","Type":"ContainerDied","Data":"87a2f073d8333c68c11b097b9d059dbb4cff13a0d1dccea8527b3fcaec17a7ae"}
Sep 08 16:07:24.132842 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.132631    2552 scope.go:117] "RemoveContainer" containerID="2abde71f4bd5ce7e75fcaebc1d862b2e390f4fcddab429161691c5f66d623dd1"
Sep 08 16:07:24.252222 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.252200    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:07:24.365342 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365313    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-istio-token\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365475 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365372    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f962868-b96c-41ee-8bc0-f038095a0085-istio-podinfo\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365475 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365411    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f962868-b96c-41ee-8bc0-f038095a0085-istiod-ca-cert\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365475 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365428    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-socket\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365657 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365476    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-credential-socket\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365657 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365515    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-certs\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365657 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365603    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-envoy\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365657 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365645    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-data\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365850 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365692    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gshm9\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-kube-api-access-gshm9\") pod \"4f962868-b96c-41ee-8bc0-f038095a0085\" (UID: \"4f962868-b96c-41ee-8bc0-f038095a0085\") "
Sep 08 16:07:24.365850 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365746    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:07:24.365959 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.365860    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f962868-b96c-41ee-8bc0-f038095a0085-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:07:24.366148 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.366103    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:07:24.366148 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.366109    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:07:24.366304 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.366154    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-data" (OuterVolumeSpecName: "istio-data") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:07:24.366304 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.366249    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f962868-b96c-41ee-8bc0-f038095a0085-istiod-ca-cert\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.366304 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.366269    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-socket\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.366304 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.366283    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-credential-socket\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.366304 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.366298    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-workload-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.367783 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.367758    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f962868-b96c-41ee-8bc0-f038095a0085-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 16:07:24.367945 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.367919    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-kube-api-access-gshm9" (OuterVolumeSpecName: "kube-api-access-gshm9") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "kube-api-access-gshm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:07:24.368051 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.368016    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:07:24.368119 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.368064    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-istio-token" (OuterVolumeSpecName: "istio-token") pod "4f962868-b96c-41ee-8bc0-f038095a0085" (UID: "4f962868-b96c-41ee-8bc0-f038095a0085"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:07:24.467785 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.467749    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-envoy\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.467785 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.467780    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f962868-b96c-41ee-8bc0-f038095a0085-istio-data\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.467785 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.467790    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gshm9\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-kube-api-access-gshm9\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.468012 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.467798    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f962868-b96c-41ee-8bc0-f038095a0085-istio-token\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:24.468012 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:24.467807    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f962868-b96c-41ee-8bc0-f038095a0085-istio-podinfo\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:25.139819 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:25.139787    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj" event={"ID":"4f962868-b96c-41ee-8bc0-f038095a0085","Type":"ContainerDied","Data":"075d9c4bff62b2b93dae8d8184ae303f8e650605f692584f49cb33b79f346bd2"}
Sep 08 16:07:25.139819 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:25.139808    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"
Sep 08 16:07:25.140319 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:25.139852    2552 scope.go:117] "RemoveContainer" containerID="87a2f073d8333c68c11b097b9d059dbb4cff13a0d1dccea8527b3fcaec17a7ae"
Sep 08 16:07:25.157620 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:25.157596    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"]
Sep 08 16:07:25.161809 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:25.161786    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5214de-openshift-default-847cc78c6b-mrxcj"]
Sep 08 16:07:27.102459 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:27.102411    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" path="/var/lib/kubelet/pods/4f962868-b96c-41ee-8bc0-f038095a0085/volumes"
Sep 08 16:07:34.858296 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:34.858258    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"]
Sep 08 16:07:34.858820 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:34.858529    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk" podUID="633caf66-6283-4fcb-8326-2aa4178059a4" containerName="payload-processing" containerID="cri-o://c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624" gracePeriod=30
Sep 08 16:07:34.869711 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:34.869658    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"]
Sep 08 16:07:34.869906 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:34.869887    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp" podUID="761ed289-225f-4a66-987b-33dc6ddef779" containerName="payload-pre-processing" containerID="cri-o://d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c" gracePeriod=30
Sep 08 16:07:35.064435 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.064375    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"]
Sep 08 16:07:35.068112 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.068079    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b8a099-gcdwf"]
Sep 08 16:07:35.105527 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.105491    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="def9d97e-16c7-4170-a18e-b11b88ef544e" path="/var/lib/kubelet/pods/def9d97e-16c7-4170-a18e-b11b88ef544e/volumes"
Sep 08 16:07:35.133008 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.132988    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:07:35.156742 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.156714    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:07:35.172258 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.172232    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633caf66-6283-4fcb-8326-2aa4178059a4-plugins-config-volume\") pod \"633caf66-6283-4fcb-8326-2aa4178059a4\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") "
Sep 08 16:07:35.172386 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.172374    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-725r5\" (UniqueName: \"kubernetes.io/projected/633caf66-6283-4fcb-8326-2aa4178059a4-kube-api-access-725r5\") pod \"633caf66-6283-4fcb-8326-2aa4178059a4\" (UID: \"633caf66-6283-4fcb-8326-2aa4178059a4\") "
Sep 08 16:07:35.172661 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.172635    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/633caf66-6283-4fcb-8326-2aa4178059a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "633caf66-6283-4fcb-8326-2aa4178059a4" (UID: "633caf66-6283-4fcb-8326-2aa4178059a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:07:35.172765 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.172739    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/633caf66-6283-4fcb-8326-2aa4178059a4-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:35.175142 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.174915    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633caf66-6283-4fcb-8326-2aa4178059a4-kube-api-access-725r5" (OuterVolumeSpecName: "kube-api-access-725r5") pod "633caf66-6283-4fcb-8326-2aa4178059a4" (UID: "633caf66-6283-4fcb-8326-2aa4178059a4"). InnerVolumeSpecName "kube-api-access-725r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:07:35.197659 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.197622    2552 generic.go:356] "Generic (PLEG): container finished" podID="633caf66-6283-4fcb-8326-2aa4178059a4" containerID="c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624" exitCode=0
Sep 08 16:07:35.197874 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.197702    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"
Sep 08 16:07:35.197874 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.197710    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk" event={"ID":"633caf66-6283-4fcb-8326-2aa4178059a4","Type":"ContainerDied","Data":"c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624"}
Sep 08 16:07:35.197874 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.197776    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk" event={"ID":"633caf66-6283-4fcb-8326-2aa4178059a4","Type":"ContainerDied","Data":"68ba8f15a083b0c5f1fb01b2be2089e01cdd6e1652cb02ec460793fc55f29609"}
Sep 08 16:07:35.197874 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.197806    2552 scope.go:117] "RemoveContainer" containerID="c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624"
Sep 08 16:07:35.200042 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.199812    2552 generic.go:356] "Generic (PLEG): container finished" podID="761ed289-225f-4a66-987b-33dc6ddef779" containerID="d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c" exitCode=0
Sep 08 16:07:35.200042 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.199961    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp" event={"ID":"761ed289-225f-4a66-987b-33dc6ddef779","Type":"ContainerDied","Data":"d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c"}
Sep 08 16:07:35.200042 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.200000    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp" event={"ID":"761ed289-225f-4a66-987b-33dc6ddef779","Type":"ContainerDied","Data":"e39cc05e06670b9d61a5966ea4a0b3bc88f6a8ed86d8f47446c3edd130eec526"}
Sep 08 16:07:35.200362 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.200085    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"
Sep 08 16:07:35.211602 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.211583    2552 scope.go:117] "RemoveContainer" containerID="c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624"
Sep 08 16:07:35.211915 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:07:35.211895    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624\": container with ID starting with c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624 not found: ID does not exist" containerID="c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624"
Sep 08 16:07:35.211966 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.211924    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624"} err="failed to get container status \"c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624\": rpc error: code = NotFound desc = could not find container \"c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624\": container with ID starting with c0f402e21818ecc2b6971f89388e7d881f93b075f43c16f747c7180ffe314624 not found: ID does not exist"
Sep 08 16:07:35.211966 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.211942    2552 scope.go:117] "RemoveContainer" containerID="d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c"
Sep 08 16:07:35.220047 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.220003    2552 scope.go:117] "RemoveContainer" containerID="d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c"
Sep 08 16:07:35.220332 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:07:35.220315    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c\": container with ID starting with d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c not found: ID does not exist" containerID="d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c"
Sep 08 16:07:35.220385 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.220338    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c"} err="failed to get container status \"d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c\": rpc error: code = NotFound desc = could not find container \"d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c\": container with ID starting with d0dd988e69d0f6b520d9246c7db7f516efb7fa2877b6838e3be030b85accf09c not found: ID does not exist"
Sep 08 16:07:35.223823 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.223798    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"]
Sep 08 16:07:35.230697 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.230671    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b8a099-664744fff5-v7vdk"]
Sep 08 16:07:35.273598 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.273565    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/761ed289-225f-4a66-987b-33dc6ddef779-plugins-config-volume\") pod \"761ed289-225f-4a66-987b-33dc6ddef779\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") "
Sep 08 16:07:35.273749 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.273653    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd4ww\" (UniqueName: \"kubernetes.io/projected/761ed289-225f-4a66-987b-33dc6ddef779-kube-api-access-zd4ww\") pod \"761ed289-225f-4a66-987b-33dc6ddef779\" (UID: \"761ed289-225f-4a66-987b-33dc6ddef779\") "
Sep 08 16:07:35.273862 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.273850    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-725r5\" (UniqueName: \"kubernetes.io/projected/633caf66-6283-4fcb-8326-2aa4178059a4-kube-api-access-725r5\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:35.273961 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.273931    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/761ed289-225f-4a66-987b-33dc6ddef779-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "761ed289-225f-4a66-987b-33dc6ddef779" (UID: "761ed289-225f-4a66-987b-33dc6ddef779"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:07:35.275619 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.275599    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/761ed289-225f-4a66-987b-33dc6ddef779-kube-api-access-zd4ww" (OuterVolumeSpecName: "kube-api-access-zd4ww") pod "761ed289-225f-4a66-987b-33dc6ddef779" (UID: "761ed289-225f-4a66-987b-33dc6ddef779"). InnerVolumeSpecName "kube-api-access-zd4ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:07:35.374857 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.374828    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd4ww\" (UniqueName: \"kubernetes.io/projected/761ed289-225f-4a66-987b-33dc6ddef779-kube-api-access-zd4ww\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:35.374857 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.374857    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/761ed289-225f-4a66-987b-33dc6ddef779-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:35.530616 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.530579    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"]
Sep 08 16:07:35.540822 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:35.540793    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b8a099-6b4d4d844b-th6sp"]
Sep 08 16:07:37.101474 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:37.101438    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="633caf66-6283-4fcb-8326-2aa4178059a4" path="/var/lib/kubelet/pods/633caf66-6283-4fcb-8326-2aa4178059a4/volumes"
Sep 08 16:07:37.101851 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:37.101815    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="761ed289-225f-4a66-987b-33dc6ddef779" path="/var/lib/kubelet/pods/761ed289-225f-4a66-987b-33dc6ddef779/volumes"
Sep 08 16:07:41.954606 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.954570    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"]
Sep 08 16:07:41.954987 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.954953    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="def9d97e-16c7-4170-a18e-b11b88ef544e" containerName="revoke-keys"
Sep 08 16:07:41.954987 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.954964    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="def9d97e-16c7-4170-a18e-b11b88ef544e" containerName="revoke-keys"
Sep 08 16:07:41.954987 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.954982    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy"
Sep 08 16:07:41.954987 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.954987    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.954999    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="761ed289-225f-4a66-987b-33dc6ddef779" containerName="payload-pre-processing"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955006    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="761ed289-225f-4a66-987b-33dc6ddef779" containerName="payload-pre-processing"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955015    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633caf66-6283-4fcb-8326-2aa4178059a4" containerName="payload-processing"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955034    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="633caf66-6283-4fcb-8326-2aa4178059a4" containerName="payload-processing"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955092    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="def9d97e-16c7-4170-a18e-b11b88ef544e" containerName="revoke-keys"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955101    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="633caf66-6283-4fcb-8326-2aa4178059a4" containerName="payload-processing"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955109    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="761ed289-225f-4a66-987b-33dc6ddef779" containerName="payload-pre-processing"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955116    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy"
Sep 08 16:07:41.955171 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.955122    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy"
Sep 08 16:07:41.958008 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.957992    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"
Sep 08 16:07:41.960794 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.960775    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:07:41.961142 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.961122    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:07:41.969765 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:41.969739    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"]
Sep 08 16:07:42.040558 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:42.040525    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ec04a5-5b70-4538-b434-7b9e2ebdb439-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2\" (UID: \"e2ec04a5-5b70-4538-b434-7b9e2ebdb439\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"
Sep 08 16:07:42.141219 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:42.141184    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ec04a5-5b70-4538-b434-7b9e2ebdb439-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2\" (UID: \"e2ec04a5-5b70-4538-b434-7b9e2ebdb439\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"
Sep 08 16:07:42.141809 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:42.141789    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ec04a5-5b70-4538-b434-7b9e2ebdb439-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2\" (UID: \"e2ec04a5-5b70-4538-b434-7b9e2ebdb439\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"
Sep 08 16:07:42.269175 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:42.269093    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"
Sep 08 16:07:42.540634 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:42.540557    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"]
Sep 08 16:07:42.543997 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:07:42.543964    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ec04a5_5b70_4538_b434_7b9e2ebdb439.slice/crio-925b6d6027f4ab04c9ddf24421e548fd81d8755c97f367e8c64c79541d883ce9 WatchSource:0}: Error finding container 925b6d6027f4ab04c9ddf24421e548fd81d8755c97f367e8c64c79541d883ce9: Status 404 returned error can't find the container with id 925b6d6027f4ab04c9ddf24421e548fd81d8755c97f367e8c64c79541d883ce9
Sep 08 16:07:43.248818 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:43.248785    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2" event={"ID":"e2ec04a5-5b70-4538-b434-7b9e2ebdb439","Type":"ContainerStarted","Data":"47779ef24e47fbb2bb83040c392d567b793e4c6d8fca6bf5a6a560baf5888dd1"}
Sep 08 16:07:43.249138 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:43.248847    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2" event={"ID":"e2ec04a5-5b70-4538-b434-7b9e2ebdb439","Type":"ContainerStarted","Data":"925b6d6027f4ab04c9ddf24421e548fd81d8755c97f367e8c64c79541d883ce9"}
Sep 08 16:07:44.255480 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:44.255447    2552 generic.go:356] "Generic (PLEG): container finished" podID="e2ec04a5-5b70-4538-b434-7b9e2ebdb439" containerID="47779ef24e47fbb2bb83040c392d567b793e4c6d8fca6bf5a6a560baf5888dd1" exitCode=0
Sep 08 16:07:44.255869 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:44.255536    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2" event={"ID":"e2ec04a5-5b70-4538-b434-7b9e2ebdb439","Type":"ContainerDied","Data":"47779ef24e47fbb2bb83040c392d567b793e4c6d8fca6bf5a6a560baf5888dd1"}
Sep 08 16:07:44.391526 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:44.391496    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"
Sep 08 16:07:44.462709 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:44.462678    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ec04a5-5b70-4538-b434-7b9e2ebdb439-maas-api-service-ca\") pod \"e2ec04a5-5b70-4538-b434-7b9e2ebdb439\" (UID: \"e2ec04a5-5b70-4538-b434-7b9e2ebdb439\") "
Sep 08 16:07:44.463056 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:44.463011    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ec04a5-5b70-4538-b434-7b9e2ebdb439-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e2ec04a5-5b70-4538-b434-7b9e2ebdb439" (UID: "e2ec04a5-5b70-4538-b434-7b9e2ebdb439"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:07:44.563867 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:44.563777    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2ec04a5-5b70-4538-b434-7b9e2ebdb439-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:07:45.262359 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:45.262332    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"
Sep 08 16:07:45.262832 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:45.262374    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2" event={"ID":"e2ec04a5-5b70-4538-b434-7b9e2ebdb439","Type":"ContainerDied","Data":"925b6d6027f4ab04c9ddf24421e548fd81d8755c97f367e8c64c79541d883ce9"}
Sep 08 16:07:45.262832 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:45.262431    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="925b6d6027f4ab04c9ddf24421e548fd81d8755c97f367e8c64c79541d883ce9"
Sep 08 16:07:46.736084 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.736055    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"]
Sep 08 16:07:46.736516 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.736501    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2ec04a5-5b70-4538-b434-7b9e2ebdb439" containerName="revoke-keys"
Sep 08 16:07:46.736562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.736519    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2ec04a5-5b70-4538-b434-7b9e2ebdb439" containerName="revoke-keys"
Sep 08 16:07:46.736562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.736537    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy"
Sep 08 16:07:46.736562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.736544    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f962868-b96c-41ee-8bc0-f038095a0085" containerName="istio-proxy"
Sep 08 16:07:46.736706 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.736612    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2ec04a5-5b70-4538-b434-7b9e2ebdb439" containerName="revoke-keys"
Sep 08 16:07:46.739158 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.739139    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.741282 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.741261    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-cf851c-openshift-default-dockercfg-q6cvn\""
Sep 08 16:07:46.752632 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.752610    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"]
Sep 08 16:07:46.884579 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884538    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-credential-socket\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.884579 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884580    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-token\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.884802 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884693    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istiod-ca-cert\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.884802 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884766    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-podinfo\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.884896 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884804    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgz2x\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-kube-api-access-bgz2x\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.884896 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884832    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-data\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.884896 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884857    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-certs\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.884896 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884877    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-envoy\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.885065 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.884900    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-socket\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.986522 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.986433    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-data\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.986766 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.986745    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-certs\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.986899 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.986885    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-envoy\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987032 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987007    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-socket\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987227 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987063    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-certs\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987227 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987074    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-credential-socket\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987227 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987109    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-token\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987227 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987191    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istiod-ca-cert\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987561 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987280    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-podinfo\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987561 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987335    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-socket\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987786 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987647    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-credential-socket\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987786 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.986793    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-data\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.987786 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.987339    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgz2x\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-kube-api-access-bgz2x\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.988390 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.988369    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istiod-ca-cert\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.989874 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.989847    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-envoy\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.990884 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.990857    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-podinfo\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.997102 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.997077    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgz2x\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-kube-api-access-bgz2x\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:46.997578 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:46.997557    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-token\") pod \"e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") " pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:47.049728 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:47.049696    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:47.326594 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:47.326562    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"]
Sep 08 16:07:47.328126 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:07:47.328081    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0ff080_ed99_4df2_b82b_eb57ff7dcf40.slice/crio-c1d99a63c6edcda5397337153b6f0d55fc054c1c2024f5e51dc74112d7df588c WatchSource:0}: Error finding container c1d99a63c6edcda5397337153b6f0d55fc054c1c2024f5e51dc74112d7df588c: Status 404 returned error can't find the container with id c1d99a63c6edcda5397337153b6f0d55fc054c1c2024f5e51dc74112d7df588c
Sep 08 16:07:47.330947 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:47.330906    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:07:47.331069 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:47.330972    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:07:47.331069 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:47.330999    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:07:48.285018 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:48.284986    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" event={"ID":"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40","Type":"ContainerStarted","Data":"1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d"}
Sep 08 16:07:48.285018 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:48.285049    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" event={"ID":"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40","Type":"ContainerStarted","Data":"c1d99a63c6edcda5397337153b6f0d55fc054c1c2024f5e51dc74112d7df588c"}
Sep 08 16:07:48.305081 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:48.305003    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podStartSLOduration=2.304984674 podStartE2EDuration="2.304984674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:07:48.303780439 +0000 UTC m=+1377.800188407" watchObservedRunningTime="2026-09-08 16:07:48.304984674 +0000 UTC m=+1377.801392642"
Sep 08 16:07:49.050277 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:49.050241    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:49.055575 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:49.055544    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:49.291787 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:49.291759    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:49.292657 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:49.292634    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:07:52.119558 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.119525    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"]
Sep 08 16:07:52.124726 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.124699    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.128233 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.128206    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-cf851c\""
Sep 08 16:07:52.128428 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.128403    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-cf851c-dockercfg-mqgjs\""
Sep 08 16:07:52.132726 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.132703    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"]
Sep 08 16:07:52.150426 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.150387    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"]
Sep 08 16:07:52.154618 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.154594    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.165217 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.165184    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"]
Sep 08 16:07:52.240849 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.239711    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.240849 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.239793    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.240849 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.239835    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4gx9\" (UniqueName: \"kubernetes.io/projected/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-kube-api-access-t4gx9\") pod \"payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.240849 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.239927    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frg8\" (UniqueName: \"kubernetes.io/projected/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-kube-api-access-6frg8\") pod \"payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.342512 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.341045    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frg8\" (UniqueName: \"kubernetes.io/projected/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-kube-api-access-6frg8\") pod \"payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.342512 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.341193    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.342512 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.341242    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.342512 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.341277    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4gx9\" (UniqueName: \"kubernetes.io/projected/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-kube-api-access-t4gx9\") pod \"payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.342512 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.342269    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.342512 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.342451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.355458 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.355424    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frg8\" (UniqueName: \"kubernetes.io/projected/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-kube-api-access-6frg8\") pod \"payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.358515 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.358488    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4gx9\" (UniqueName: \"kubernetes.io/projected/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-kube-api-access-t4gx9\") pod \"payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.438506 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.438413    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:52.467854 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.467765    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:52.567234 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.567200    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"]
Sep 08 16:07:52.569689 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.569664    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-13d3a5-rfmj2"]
Sep 08 16:07:52.729638 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.729613    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"]
Sep 08 16:07:52.730622 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:07:52.730598    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d44de2e_28e7_45c5_b2a6_c56d71d01f5b.slice/crio-749301b2d99589e6da964c292b09efc5aad2764ee595b7fa934722dd78b9b42d WatchSource:0}: Error finding container 749301b2d99589e6da964c292b09efc5aad2764ee595b7fa934722dd78b9b42d: Status 404 returned error can't find the container with id 749301b2d99589e6da964c292b09efc5aad2764ee595b7fa934722dd78b9b42d
Sep 08 16:07:52.758550 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:52.758525    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"]
Sep 08 16:07:52.766239 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:07:52.766214    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod383040d6_75c0_4de7_b4a7_63e7cd13e9ab.slice/crio-8c9c7ab4422fb5b3d0633910e6e221f2fd84caad39af617e7cb53fe5d2bb905f WatchSource:0}: Error finding container 8c9c7ab4422fb5b3d0633910e6e221f2fd84caad39af617e7cb53fe5d2bb905f: Status 404 returned error can't find the container with id 8c9c7ab4422fb5b3d0633910e6e221f2fd84caad39af617e7cb53fe5d2bb905f
Sep 08 16:07:53.101263 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.101180    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2ec04a5-5b70-4538-b434-7b9e2ebdb439" path="/var/lib/kubelet/pods/e2ec04a5-5b70-4538-b434-7b9e2ebdb439/volumes"
Sep 08 16:07:53.319365 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.319317    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt" event={"ID":"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b","Type":"ContainerStarted","Data":"b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd"}
Sep 08 16:07:53.319850 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.319400    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:07:53.319850 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.319429    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt" event={"ID":"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b","Type":"ContainerStarted","Data":"749301b2d99589e6da964c292b09efc5aad2764ee595b7fa934722dd78b9b42d"}
Sep 08 16:07:53.320964 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.320940    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv" event={"ID":"383040d6-75c0-4de7-b4a7-63e7cd13e9ab","Type":"ContainerStarted","Data":"6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b"}
Sep 08 16:07:53.321100 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.320985    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv" event={"ID":"383040d6-75c0-4de7-b4a7-63e7cd13e9ab","Type":"ContainerStarted","Data":"8c9c7ab4422fb5b3d0633910e6e221f2fd84caad39af617e7cb53fe5d2bb905f"}
Sep 08 16:07:53.321100 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.321017    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:07:53.351531 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.351430    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv" podStartSLOduration=1.351411991 podStartE2EDuration="1.351411991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:07:53.349987545 +0000 UTC m=+1382.846395523" watchObservedRunningTime="2026-09-08 16:07:53.351411991 +0000 UTC m=+1382.847819961"
Sep 08 16:07:53.352955 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:53.352920    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt" podStartSLOduration=1.352910115 podStartE2EDuration="1.352910115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:07:53.334896032 +0000 UTC m=+1382.831304000" watchObservedRunningTime="2026-09-08 16:07:53.352910115 +0000 UTC m=+1382.849318128"
Sep 08 16:07:59.590792 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.590758    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"]
Sep 08 16:07:59.595398 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.595382    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"
Sep 08 16:07:59.597651 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.597630    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:07:59.597773 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.597636    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:07:59.603477 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.603414    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"]
Sep 08 16:07:59.715798 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.715748    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a701a1a2-b751-46f5-ab1f-ff4dec801042-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82\" (UID: \"a701a1a2-b751-46f5-ab1f-ff4dec801042\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"
Sep 08 16:07:59.817013 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.816972    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a701a1a2-b751-46f5-ab1f-ff4dec801042-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82\" (UID: \"a701a1a2-b751-46f5-ab1f-ff4dec801042\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"
Sep 08 16:07:59.817641 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.817619    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a701a1a2-b751-46f5-ab1f-ff4dec801042-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82\" (UID: \"a701a1a2-b751-46f5-ab1f-ff4dec801042\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"
Sep 08 16:07:59.908001 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:07:59.907967    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"
Sep 08 16:08:00.230421 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:00.230217    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"]
Sep 08 16:08:00.370045 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:00.369997    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82" event={"ID":"a701a1a2-b751-46f5-ab1f-ff4dec801042","Type":"ContainerStarted","Data":"1b5dc807c3e1f1d9d6a9d45ca89892409df7e6b4d2e9209f0f2c3bba007159ec"}
Sep 08 16:08:01.377545 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:01.377510    2552 generic.go:356] "Generic (PLEG): container finished" podID="a701a1a2-b751-46f5-ab1f-ff4dec801042" containerID="06d2d90b40cbb6e02648ac8c9f51a93352c141b534aba29cf828479ee7d72c8c" exitCode=0
Sep 08 16:08:01.377930 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:01.377600    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82" event={"ID":"a701a1a2-b751-46f5-ab1f-ff4dec801042","Type":"ContainerDied","Data":"06d2d90b40cbb6e02648ac8c9f51a93352c141b534aba29cf828479ee7d72c8c"}
Sep 08 16:08:02.563808 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:02.563782    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"
Sep 08 16:08:02.656457 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:02.656365    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a701a1a2-b751-46f5-ab1f-ff4dec801042-maas-api-service-ca\") pod \"a701a1a2-b751-46f5-ab1f-ff4dec801042\" (UID: \"a701a1a2-b751-46f5-ab1f-ff4dec801042\") "
Sep 08 16:08:02.656697 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:02.656675    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a701a1a2-b751-46f5-ab1f-ff4dec801042-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a701a1a2-b751-46f5-ab1f-ff4dec801042" (UID: "a701a1a2-b751-46f5-ab1f-ff4dec801042"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:08:02.757634 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:02.757602    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a701a1a2-b751-46f5-ab1f-ff4dec801042-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:03.390706 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:03.390675    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"
Sep 08 16:08:03.390879 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:03.390678    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82" event={"ID":"a701a1a2-b751-46f5-ab1f-ff4dec801042","Type":"ContainerDied","Data":"1b5dc807c3e1f1d9d6a9d45ca89892409df7e6b4d2e9209f0f2c3bba007159ec"}
Sep 08 16:08:03.390879 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:03.390821    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b5dc807c3e1f1d9d6a9d45ca89892409df7e6b4d2e9209f0f2c3bba007159ec"
Sep 08 16:08:04.328066 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:04.328017    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:08:04.328517 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:04.328109    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:08:05.334191 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:05.334143    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 16:08:05.334682 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:05.334198    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 16:08:06.408138 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:06.408100    2552 generic.go:356] "Generic (PLEG): container finished" podID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerID="1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d" exitCode=0
Sep 08 16:08:06.408536 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:06.408172    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" event={"ID":"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40","Type":"ContainerDied","Data":"1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d"}
Sep 08 16:08:06.408626 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:06.408573    2552 scope.go:117] "RemoveContainer" containerID="1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d"
Sep 08 16:08:06.409605 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:06.409579    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:08:06.409655 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:06.409645    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:08:06.409694 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:06.409675    2552 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 16:08:07.415996 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:07.415963    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" event={"ID":"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40","Type":"ContainerStarted","Data":"56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872"}
Sep 08 16:08:08.050637 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:08.050598    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:08:08.050853 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:08.050829    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 16:08:08.050953 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:08.050892    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 16:08:09.050201 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:09.050162    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 16:08:09.050796 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:09.050758    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 16:08:10.050166 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:10.050133    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 16:08:10.050392 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:10.050257    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 16:08:10.390895 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:10.390865    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"]
Sep 08 16:08:10.397103 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:10.397081    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e69292-vcz82"]
Sep 08 16:08:11.051103 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:11.050591    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 16:08:11.051103 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:11.050656    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 16:08:11.103184 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:11.103143    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a701a1a2-b751-46f5-ab1f-ff4dec801042" path="/var/lib/kubelet/pods/a701a1a2-b751-46f5-ab1f-ff4dec801042/volumes"
Sep 08 16:08:12.050520 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:12.050466    2552 patch_prober.go:28] interesting pod/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 08 16:08:12.050733 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:12.050541    2552 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 08 16:08:13.053770 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:13.053739    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:08:13.054277 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:13.054114    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:08:13.054808 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:13.054791    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:08:17.667241 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:17.667202    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"]
Sep 08 16:08:17.667837 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:17.667448    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy" containerID="cri-o://e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e" gracePeriod=30
Sep 08 16:08:22.921884 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.921860    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:08:22.980650 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980572    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f3e7e73-3454-42d9-a380-8e47787bee59-istiod-ca-cert\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.980650 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980644    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-token\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.980885 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980703    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6gsd\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-kube-api-access-g6gsd\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.980885 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980741    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-data\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.980885 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980772    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-envoy\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.980885 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980797    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-certs\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.980885 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980848    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-socket\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.980885 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980882    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-credential-socket\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.981243 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980909    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-podinfo\") pod \"3f3e7e73-3454-42d9-a380-8e47787bee59\" (UID: \"3f3e7e73-3454-42d9-a380-8e47787bee59\") "
Sep 08 16:08:22.981243 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.980941    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f3e7e73-3454-42d9-a380-8e47787bee59-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:08:22.981243 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.981132    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:08:22.981243 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.981217    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:08:22.981424 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.981355    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f3e7e73-3454-42d9-a380-8e47787bee59-istiod-ca-cert\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:22.981424 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.981376    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-data\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:22.981424 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.981389    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-socket\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:22.981620 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.981597    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:08:22.981620 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.981614    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:08:22.983472 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.983450    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-kube-api-access-g6gsd" (OuterVolumeSpecName: "kube-api-access-g6gsd") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "kube-api-access-g6gsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:08:22.983554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.983456    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:08:22.983554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.983491    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:08:22.983554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:22.983491    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f3e7e73-3454-42d9-a380-8e47787bee59" (UID: "3f3e7e73-3454-42d9-a380-8e47787bee59"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 16:08:23.082798 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.082767    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-token\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:23.082798 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.082791    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6gsd\" (UniqueName: \"kubernetes.io/projected/3f3e7e73-3454-42d9-a380-8e47787bee59-kube-api-access-g6gsd\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:23.082798 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.082801    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-envoy\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:23.083043 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.082809    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-workload-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:23.083043 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.082817    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f3e7e73-3454-42d9-a380-8e47787bee59-credential-socket\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:23.083043 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.082825    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f3e7e73-3454-42d9-a380-8e47787bee59-istio-podinfo\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:08:23.534575 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.534535    2552 generic.go:356] "Generic (PLEG): container finished" podID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerID="e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e" exitCode=0
Sep 08 16:08:23.534767 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.534608    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"
Sep 08 16:08:23.534767 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.534618    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" event={"ID":"3f3e7e73-3454-42d9-a380-8e47787bee59","Type":"ContainerDied","Data":"e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e"}
Sep 08 16:08:23.534767 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.534699    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr" event={"ID":"3f3e7e73-3454-42d9-a380-8e47787bee59","Type":"ContainerDied","Data":"f2193df1d028d1aee325160fba63eaa50af392beb5949027ec4bc911a252f2d3"}
Sep 08 16:08:23.534767 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.534730    2552 scope.go:117] "RemoveContainer" containerID="e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e"
Sep 08 16:08:23.544770 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.544748    2552 scope.go:117] "RemoveContainer" containerID="26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442"
Sep 08 16:08:23.553884 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.553862    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"]
Sep 08 16:08:23.554419 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.554402    2552 scope.go:117] "RemoveContainer" containerID="e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e"
Sep 08 16:08:23.554716 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:08:23.554692    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e\": container with ID starting with e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e not found: ID does not exist" containerID="e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e"
Sep 08 16:08:23.554819 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.554725    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e"} err="failed to get container status \"e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e\": rpc error: code = NotFound desc = could not find container \"e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e\": container with ID starting with e9464ac2a61dfac1b263e577fb4467091c6cb1044d87f4e46487aa79efe6038e not found: ID does not exist"
Sep 08 16:08:23.554819 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.554742    2552 scope.go:117] "RemoveContainer" containerID="26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442"
Sep 08 16:08:23.555180 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:08:23.555042    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442\": container with ID starting with 26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442 not found: ID does not exist" containerID="26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442"
Sep 08 16:08:23.555180 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.555076    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442"} err="failed to get container status \"26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442\": rpc error: code = NotFound desc = could not find container \"26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442\": container with ID starting with 26e1e5d65f23febb93654dd030a24d1a3b3585ac5220fbba12ea591c8253d442 not found: ID does not exist"
Sep 08 16:08:23.558378 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:23.558297    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e69292-openshift-default-cd9fc6dc4-xwzcr"]
Sep 08 16:08:25.101082 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:25.101018    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" path="/var/lib/kubelet/pods/3f3e7e73-3454-42d9-a380-8e47787bee59/volumes"
Sep 08 16:08:49.706081 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:49.705988    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9g54m_26b382fb-6099-4697-91f1-c4d30851a597/payload-processing/0.log"
Sep 08 16:08:51.222641 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:08:51.222607    2552 scope.go:117] "RemoveContainer" containerID="f157045bf31a510c36a861e9510e44e98a54cc8ae5b89594596f504b330f003d"
Sep 08 16:09:30.637825 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:30.637787    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv_383040d6-75c0-4de7-b4a7-63e7cd13e9ab/payload-processing/0.log"
Sep 08 16:09:30.756883 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:30.756844    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9g54m_26b382fb-6099-4697-91f1-c4d30851a597/payload-processing/0.log"
Sep 08 16:09:36.674907 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.674870    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"]
Sep 08 16:09:36.675382 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675289    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy"
Sep 08 16:09:36.675382 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675302    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy"
Sep 08 16:09:36.675382 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675315    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a701a1a2-b751-46f5-ab1f-ff4dec801042" containerName="revoke-keys"
Sep 08 16:09:36.675382 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675320    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a701a1a2-b751-46f5-ab1f-ff4dec801042" containerName="revoke-keys"
Sep 08 16:09:36.675382 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675337    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy"
Sep 08 16:09:36.675382 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675349    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy"
Sep 08 16:09:36.675578 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675420    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a701a1a2-b751-46f5-ab1f-ff4dec801042" containerName="revoke-keys"
Sep 08 16:09:36.675578 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.675429    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy"
Sep 08 16:09:36.677436 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.677420    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:36.681528 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.681501    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-a2951a-dockercfg-74gzf\""
Sep 08 16:09:36.681728 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.681558    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-a2951a\""
Sep 08 16:09:36.690468 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.690443    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"]
Sep 08 16:09:36.700570 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.700543    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttwt\" (UniqueName: \"kubernetes.io/projected/993e46ec-3606-4e33-a391-da3cd38f32ea-kube-api-access-zttwt\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:36.700727 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.700708    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993e46ec-3606-4e33-a391-da3cd38f32ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:36.802042 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.802000    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993e46ec-3606-4e33-a391-da3cd38f32ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:36.802211 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.802112    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttwt\" (UniqueName: \"kubernetes.io/projected/993e46ec-3606-4e33-a391-da3cd38f32ea-kube-api-access-zttwt\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:36.802680 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.802659    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993e46ec-3606-4e33-a391-da3cd38f32ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:36.811695 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.811664    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttwt\" (UniqueName: \"kubernetes.io/projected/993e46ec-3606-4e33-a391-da3cd38f32ea-kube-api-access-zttwt\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:36.988745 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:36.988656    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:37.259673 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:37.259642    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"]
Sep 08 16:09:37.261848 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:09:37.261820    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993e46ec_3606_4e33_a391_da3cd38f32ea.slice/crio-4234fb6255eae8d3e242f70677efec2d14c430da02da2846bda723b1c6194003 WatchSource:0}: Error finding container 4234fb6255eae8d3e242f70677efec2d14c430da02da2846bda723b1c6194003: Status 404 returned error can't find the container with id 4234fb6255eae8d3e242f70677efec2d14c430da02da2846bda723b1c6194003
Sep 08 16:09:37.947578 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:37.947543    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l" event={"ID":"993e46ec-3606-4e33-a391-da3cd38f32ea","Type":"ContainerStarted","Data":"cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23"}
Sep 08 16:09:37.947992 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:37.947592    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l" event={"ID":"993e46ec-3606-4e33-a391-da3cd38f32ea","Type":"ContainerStarted","Data":"4234fb6255eae8d3e242f70677efec2d14c430da02da2846bda723b1c6194003"}
Sep 08 16:09:37.947992 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:37.947738    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:37.965485 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:37.965435    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l" podStartSLOduration=1.965415856 podStartE2EDuration="1.965415856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:09:37.965390462 +0000 UTC m=+1487.461798455" watchObservedRunningTime="2026-09-08 16:09:37.965415856 +0000 UTC m=+1487.461823824"
Sep 08 16:09:48.955105 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:48.955074    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:09:51.246858 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:09:51.246825    2552 scope.go:117] "RemoveContainer" containerID="9a01acdaf30999accfa52c30f467e3a430268895fdd5c61cbdc093677f123b68"
Sep 08 16:10:00.451183 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.451142    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"]
Sep 08 16:10:00.451931 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.451910    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f3e7e73-3454-42d9-a380-8e47787bee59" containerName="istio-proxy"
Sep 08 16:10:00.455339 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.455319    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"
Sep 08 16:10:00.457717 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.457649    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:10:00.458626 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.458598    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:10:00.461656 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.461633    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"]
Sep 08 16:10:00.525402 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.525366    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fefee72-5524-486c-ad10-fcf356f1c607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc\" (UID: \"6fefee72-5524-486c-ad10-fcf356f1c607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"
Sep 08 16:10:00.626731 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.626690    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fefee72-5524-486c-ad10-fcf356f1c607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc\" (UID: \"6fefee72-5524-486c-ad10-fcf356f1c607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"
Sep 08 16:10:00.627435 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.627408    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fefee72-5524-486c-ad10-fcf356f1c607-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc\" (UID: \"6fefee72-5524-486c-ad10-fcf356f1c607\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"
Sep 08 16:10:00.768507 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:00.768425    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"
Sep 08 16:10:01.041109 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:01.041079    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"]
Sep 08 16:10:01.043244 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:10:01.043213    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fefee72_5524_486c_ad10_fcf356f1c607.slice/crio-c6e9b454468e841b3cfb6db86d7887400bc24a7e672b503f50efd010e514fde0 WatchSource:0}: Error finding container c6e9b454468e841b3cfb6db86d7887400bc24a7e672b503f50efd010e514fde0: Status 404 returned error can't find the container with id c6e9b454468e841b3cfb6db86d7887400bc24a7e672b503f50efd010e514fde0
Sep 08 16:10:01.077568 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:01.077535    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc" event={"ID":"6fefee72-5524-486c-ad10-fcf356f1c607","Type":"ContainerStarted","Data":"c6e9b454468e841b3cfb6db86d7887400bc24a7e672b503f50efd010e514fde0"}
Sep 08 16:10:02.084770 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:02.084680    2552 generic.go:356] "Generic (PLEG): container finished" podID="6fefee72-5524-486c-ad10-fcf356f1c607" containerID="f929730c509f2fb78a178c653452f9ebd81199315c4b850b086d6ed2ce58ec53" exitCode=0
Sep 08 16:10:02.085132 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:02.084767    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc" event={"ID":"6fefee72-5524-486c-ad10-fcf356f1c607","Type":"ContainerDied","Data":"f929730c509f2fb78a178c653452f9ebd81199315c4b850b086d6ed2ce58ec53"}
Sep 08 16:10:03.224754 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:03.224729    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"
Sep 08 16:10:03.352899 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:03.352811    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fefee72-5524-486c-ad10-fcf356f1c607-maas-api-service-ca\") pod \"6fefee72-5524-486c-ad10-fcf356f1c607\" (UID: \"6fefee72-5524-486c-ad10-fcf356f1c607\") "
Sep 08 16:10:03.353242 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:03.353219    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fefee72-5524-486c-ad10-fcf356f1c607-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6fefee72-5524-486c-ad10-fcf356f1c607" (UID: "6fefee72-5524-486c-ad10-fcf356f1c607"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:10:03.453619 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:03.453579    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fefee72-5524-486c-ad10-fcf356f1c607-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:04.096677 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:04.096646    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"
Sep 08 16:10:04.096880 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:04.096694    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc" event={"ID":"6fefee72-5524-486c-ad10-fcf356f1c607","Type":"ContainerDied","Data":"c6e9b454468e841b3cfb6db86d7887400bc24a7e672b503f50efd010e514fde0"}
Sep 08 16:10:04.096880 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:04.096739    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6e9b454468e841b3cfb6db86d7887400bc24a7e672b503f50efd010e514fde0"
Sep 08 16:10:06.531810 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.531779    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"]
Sep 08 16:10:06.532705 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.532671    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l" podUID="993e46ec-3606-4e33-a391-da3cd38f32ea" containerName="payload-pre-processing" containerID="cri-o://cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23" gracePeriod=30
Sep 08 16:10:06.674112 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.674081    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"]
Sep 08 16:10:06.676872 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.676845    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-a2951a-6wpnc"]
Sep 08 16:10:06.795473 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.795403    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:10:06.887315 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.887283    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993e46ec-3606-4e33-a391-da3cd38f32ea-plugins-config-volume\") pod \"993e46ec-3606-4e33-a391-da3cd38f32ea\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") "
Sep 08 16:10:06.887498 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.887380    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zttwt\" (UniqueName: \"kubernetes.io/projected/993e46ec-3606-4e33-a391-da3cd38f32ea-kube-api-access-zttwt\") pod \"993e46ec-3606-4e33-a391-da3cd38f32ea\" (UID: \"993e46ec-3606-4e33-a391-da3cd38f32ea\") "
Sep 08 16:10:06.887747 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.887708    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/993e46ec-3606-4e33-a391-da3cd38f32ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "993e46ec-3606-4e33-a391-da3cd38f32ea" (UID: "993e46ec-3606-4e33-a391-da3cd38f32ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:10:06.887825 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.887781    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/993e46ec-3606-4e33-a391-da3cd38f32ea-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:06.889442 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.889423    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993e46ec-3606-4e33-a391-da3cd38f32ea-kube-api-access-zttwt" (OuterVolumeSpecName: "kube-api-access-zttwt") pod "993e46ec-3606-4e33-a391-da3cd38f32ea" (UID: "993e46ec-3606-4e33-a391-da3cd38f32ea"). InnerVolumeSpecName "kube-api-access-zttwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:10:06.989054 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:06.989006    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zttwt\" (UniqueName: \"kubernetes.io/projected/993e46ec-3606-4e33-a391-da3cd38f32ea-kube-api-access-zttwt\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:07.101348 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.101252    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fefee72-5524-486c-ad10-fcf356f1c607" path="/var/lib/kubelet/pods/6fefee72-5524-486c-ad10-fcf356f1c607/volumes"
Sep 08 16:10:07.114277 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.114252    2552 generic.go:356] "Generic (PLEG): container finished" podID="993e46ec-3606-4e33-a391-da3cd38f32ea" containerID="cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23" exitCode=0
Sep 08 16:10:07.114417 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.114287    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l" event={"ID":"993e46ec-3606-4e33-a391-da3cd38f32ea","Type":"ContainerDied","Data":"cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23"}
Sep 08 16:10:07.114417 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.114294    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"
Sep 08 16:10:07.114417 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.114335    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l" event={"ID":"993e46ec-3606-4e33-a391-da3cd38f32ea","Type":"ContainerDied","Data":"4234fb6255eae8d3e242f70677efec2d14c430da02da2846bda723b1c6194003"}
Sep 08 16:10:07.114417 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.114357    2552 scope.go:117] "RemoveContainer" containerID="cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23"
Sep 08 16:10:07.123185 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.123163    2552 scope.go:117] "RemoveContainer" containerID="cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23"
Sep 08 16:10:07.123483 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:10:07.123462    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23\": container with ID starting with cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23 not found: ID does not exist" containerID="cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23"
Sep 08 16:10:07.123586 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.123490    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23"} err="failed to get container status \"cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23\": rpc error: code = NotFound desc = could not find container \"cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23\": container with ID starting with cbe34c286ba65c2c5979c13a174c653ab7551214b961e357f55ca3a49dee1a23 not found: ID does not exist"
Sep 08 16:10:07.134605 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.134574    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"]
Sep 08 16:10:07.138273 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:07.138248    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2951a-6d87c6d486-hgl8l"]
Sep 08 16:10:09.101305 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:09.101271    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="993e46ec-3606-4e33-a391-da3cd38f32ea" path="/var/lib/kubelet/pods/993e46ec-3606-4e33-a391-da3cd38f32ea/volumes"
Sep 08 16:10:16.964096 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.963955    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"]
Sep 08 16:10:16.964480 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.964456    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fefee72-5524-486c-ad10-fcf356f1c607" containerName="revoke-keys"
Sep 08 16:10:16.964480 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.964470    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fefee72-5524-486c-ad10-fcf356f1c607" containerName="revoke-keys"
Sep 08 16:10:16.964554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.964499    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="993e46ec-3606-4e33-a391-da3cd38f32ea" containerName="payload-pre-processing"
Sep 08 16:10:16.964554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.964505    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="993e46ec-3606-4e33-a391-da3cd38f32ea" containerName="payload-pre-processing"
Sep 08 16:10:16.964616 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.964589    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="993e46ec-3606-4e33-a391-da3cd38f32ea" containerName="payload-pre-processing"
Sep 08 16:10:16.964616 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.964601    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fefee72-5524-486c-ad10-fcf356f1c607" containerName="revoke-keys"
Sep 08 16:10:16.967565 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.967548    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"
Sep 08 16:10:16.969818 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.969796    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:10:16.969950 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.969838    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:10:16.975666 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.975642    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"]
Sep 08 16:10:16.995001 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:16.994969    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ae7c-33d9-4526-a930-b716ffae7113-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c\" (UID: \"4e90ae7c-33d9-4526-a930-b716ffae7113\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"
Sep 08 16:10:17.096241 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:17.096207    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ae7c-33d9-4526-a930-b716ffae7113-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c\" (UID: \"4e90ae7c-33d9-4526-a930-b716ffae7113\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"
Sep 08 16:10:17.097019 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:17.096996    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ae7c-33d9-4526-a930-b716ffae7113-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c\" (UID: \"4e90ae7c-33d9-4526-a930-b716ffae7113\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"
Sep 08 16:10:17.280899 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:17.280819    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"
Sep 08 16:10:17.556349 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:17.556325    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"]
Sep 08 16:10:17.558552 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:10:17.558523    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e90ae7c_33d9_4526_a930_b716ffae7113.slice/crio-2573b699638a1d30e4cbb7187308d7e7979ea6abbd7f976bd23eee8ccadb71a4 WatchSource:0}: Error finding container 2573b699638a1d30e4cbb7187308d7e7979ea6abbd7f976bd23eee8ccadb71a4: Status 404 returned error can't find the container with id 2573b699638a1d30e4cbb7187308d7e7979ea6abbd7f976bd23eee8ccadb71a4
Sep 08 16:10:18.177370 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:18.177339    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c" event={"ID":"4e90ae7c-33d9-4526-a930-b716ffae7113","Type":"ContainerStarted","Data":"d723732111615e452bed41c258fff4066169539067de35dd10e97c3e4b4e67c8"}
Sep 08 16:10:18.177743 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:18.177386    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c" event={"ID":"4e90ae7c-33d9-4526-a930-b716ffae7113","Type":"ContainerStarted","Data":"2573b699638a1d30e4cbb7187308d7e7979ea6abbd7f976bd23eee8ccadb71a4"}
Sep 08 16:10:18.194946 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:18.194898    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c" podStartSLOduration=2.194883313 podStartE2EDuration="2.194883313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:10:18.192725777 +0000 UTC m=+1527.689133759" watchObservedRunningTime="2026-09-08 16:10:18.194883313 +0000 UTC m=+1527.691291281"
Sep 08 16:10:19.184168 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:19.184123    2552 generic.go:356] "Generic (PLEG): container finished" podID="4e90ae7c-33d9-4526-a930-b716ffae7113" containerID="d723732111615e452bed41c258fff4066169539067de35dd10e97c3e4b4e67c8" exitCode=0
Sep 08 16:10:19.184631 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:19.184208    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c" event={"ID":"4e90ae7c-33d9-4526-a930-b716ffae7113","Type":"ContainerDied","Data":"d723732111615e452bed41c258fff4066169539067de35dd10e97c3e4b4e67c8"}
Sep 08 16:10:20.319236 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:20.319212    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"
Sep 08 16:10:20.431519 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:20.431485    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ae7c-33d9-4526-a930-b716ffae7113-maas-api-service-ca\") pod \"4e90ae7c-33d9-4526-a930-b716ffae7113\" (UID: \"4e90ae7c-33d9-4526-a930-b716ffae7113\") "
Sep 08 16:10:20.431828 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:20.431804    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e90ae7c-33d9-4526-a930-b716ffae7113-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e90ae7c-33d9-4526-a930-b716ffae7113" (UID: "4e90ae7c-33d9-4526-a930-b716ffae7113"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:10:20.532703 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:20.532622    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ae7c-33d9-4526-a930-b716ffae7113-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:21.196279 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:21.196249    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"
Sep 08 16:10:21.196477 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:21.196286    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c" event={"ID":"4e90ae7c-33d9-4526-a930-b716ffae7113","Type":"ContainerDied","Data":"2573b699638a1d30e4cbb7187308d7e7979ea6abbd7f976bd23eee8ccadb71a4"}
Sep 08 16:10:21.196477 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:21.196336    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2573b699638a1d30e4cbb7187308d7e7979ea6abbd7f976bd23eee8ccadb71a4"
Sep 08 16:10:32.566096 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.566062    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"]
Sep 08 16:10:32.566491 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.566327    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv" podUID="383040d6-75c0-4de7-b4a7-63e7cd13e9ab" containerName="payload-processing" containerID="cri-o://6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b" gracePeriod=30
Sep 08 16:10:32.574316 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.574284    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"]
Sep 08 16:10:32.574784 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.574727    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt" podUID="0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" containerName="payload-pre-processing" containerID="cri-o://b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd" gracePeriod=30
Sep 08 16:10:32.731447 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.731405    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"]
Sep 08 16:10:32.733468 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.733438    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cf851c-c8m6c"]
Sep 08 16:10:32.853705 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.853681    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:10:32.856824 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.856806    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:10:32.952346 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.952316    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-plugins-config-volume\") pod \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") "
Sep 08 16:10:32.952537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.952405    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4gx9\" (UniqueName: \"kubernetes.io/projected/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-kube-api-access-t4gx9\") pod \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") "
Sep 08 16:10:32.952537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.952432    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6frg8\" (UniqueName: \"kubernetes.io/projected/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-kube-api-access-6frg8\") pod \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\" (UID: \"383040d6-75c0-4de7-b4a7-63e7cd13e9ab\") "
Sep 08 16:10:32.952537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.952519    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-plugins-config-volume\") pod \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\" (UID: \"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b\") "
Sep 08 16:10:32.952714 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.952659    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "383040d6-75c0-4de7-b4a7-63e7cd13e9ab" (UID: "383040d6-75c0-4de7-b4a7-63e7cd13e9ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:10:32.952889 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.952870    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:32.952985 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.952895    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" (UID: "0d44de2e-28e7-45c5-b2a6-c56d71d01f5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:10:32.954593 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.954572    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-kube-api-access-6frg8" (OuterVolumeSpecName: "kube-api-access-6frg8") pod "383040d6-75c0-4de7-b4a7-63e7cd13e9ab" (UID: "383040d6-75c0-4de7-b4a7-63e7cd13e9ab"). InnerVolumeSpecName "kube-api-access-6frg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:10:32.954593 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:32.954583    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-kube-api-access-t4gx9" (OuterVolumeSpecName: "kube-api-access-t4gx9") pod "0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" (UID: "0d44de2e-28e7-45c5-b2a6-c56d71d01f5b"). InnerVolumeSpecName "kube-api-access-t4gx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:10:33.053511 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.053475    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-plugins-config-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:33.053511 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.053505    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4gx9\" (UniqueName: \"kubernetes.io/projected/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b-kube-api-access-t4gx9\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:33.053511 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.053514    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6frg8\" (UniqueName: \"kubernetes.io/projected/383040d6-75c0-4de7-b4a7-63e7cd13e9ab-kube-api-access-6frg8\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:10:33.108267 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.108179    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e90ae7c-33d9-4526-a930-b716ffae7113" path="/var/lib/kubelet/pods/4e90ae7c-33d9-4526-a930-b716ffae7113/volumes"
Sep 08 16:10:33.262554 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.262525    2552 generic.go:356] "Generic (PLEG): container finished" podID="383040d6-75c0-4de7-b4a7-63e7cd13e9ab" containerID="6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b" exitCode=0
Sep 08 16:10:33.262734 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.262578    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"
Sep 08 16:10:33.262734 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.262613    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv" event={"ID":"383040d6-75c0-4de7-b4a7-63e7cd13e9ab","Type":"ContainerDied","Data":"6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b"}
Sep 08 16:10:33.262734 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.262665    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv" event={"ID":"383040d6-75c0-4de7-b4a7-63e7cd13e9ab","Type":"ContainerDied","Data":"8c9c7ab4422fb5b3d0633910e6e221f2fd84caad39af617e7cb53fe5d2bb905f"}
Sep 08 16:10:33.262734 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.262678    2552 scope.go:117] "RemoveContainer" containerID="6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b"
Sep 08 16:10:33.264196 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.264154    2552 generic.go:356] "Generic (PLEG): container finished" podID="0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" containerID="b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd" exitCode=0
Sep 08 16:10:33.264331 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.264207    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"
Sep 08 16:10:33.264331 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.264217    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt" event={"ID":"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b","Type":"ContainerDied","Data":"b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd"}
Sep 08 16:10:33.264331 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.264259    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt" event={"ID":"0d44de2e-28e7-45c5-b2a6-c56d71d01f5b","Type":"ContainerDied","Data":"749301b2d99589e6da964c292b09efc5aad2764ee595b7fa934722dd78b9b42d"}
Sep 08 16:10:33.271891 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.271874    2552 scope.go:117] "RemoveContainer" containerID="6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b"
Sep 08 16:10:33.272193 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:10:33.272173    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b\": container with ID starting with 6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b not found: ID does not exist" containerID="6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b"
Sep 08 16:10:33.272268 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.272200    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b"} err="failed to get container status \"6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b\": rpc error: code = NotFound desc = could not find container \"6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b\": container with ID starting with 6bce02bb5197652c3439150c6f1a12b84f7fbac3d57b4287b2e902abae55a48b not found: ID does not exist"
Sep 08 16:10:33.272268 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.272218    2552 scope.go:117] "RemoveContainer" containerID="b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd"
Sep 08 16:10:33.280910 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.280894    2552 scope.go:117] "RemoveContainer" containerID="b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd"
Sep 08 16:10:33.281185 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:10:33.281163    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd\": container with ID starting with b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd not found: ID does not exist" containerID="b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd"
Sep 08 16:10:33.281256 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.281190    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd"} err="failed to get container status \"b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd\": rpc error: code = NotFound desc = could not find container \"b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd\": container with ID starting with b4d7db0ab62415ddac416d025e6dd51b3f56a978b07e881efc57495c6cd78abd not found: ID does not exist"
Sep 08 16:10:33.282190 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.282171    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"]
Sep 08 16:10:33.285513 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.285486    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-cf851c-56465fb8f-jknjv"]
Sep 08 16:10:33.296174 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.296152    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"]
Sep 08 16:10:33.297902 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:33.297883    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cf851c-698c57dd9d-4mpxt"]
Sep 08 16:10:35.101276 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:35.101237    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" path="/var/lib/kubelet/pods/0d44de2e-28e7-45c5-b2a6-c56d71d01f5b/volumes"
Sep 08 16:10:35.101773 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:35.101751    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="383040d6-75c0-4de7-b4a7-63e7cd13e9ab" path="/var/lib/kubelet/pods/383040d6-75c0-4de7-b4a7-63e7cd13e9ab/volumes"
Sep 08 16:10:51.267580 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:10:51.267550    2552 scope.go:117] "RemoveContainer" containerID="71c19550f6bb65f6cf7dfc009cd7b92a1d70fc32817053146b604c8f2c866390"
Sep 08 16:11:21.104101 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.104070    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"]
Sep 08 16:11:21.104703 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.104295    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy" containerID="cri-o://56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872" gracePeriod=30
Sep 08 16:11:21.480993 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.480958    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"]
Sep 08 16:11:21.481386 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481368    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" containerName="payload-pre-processing"
Sep 08 16:11:21.481386 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481383    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" containerName="payload-pre-processing"
Sep 08 16:11:21.481537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481393    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e90ae7c-33d9-4526-a930-b716ffae7113" containerName="revoke-keys"
Sep 08 16:11:21.481537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481399    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e90ae7c-33d9-4526-a930-b716ffae7113" containerName="revoke-keys"
Sep 08 16:11:21.481537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481419    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="383040d6-75c0-4de7-b4a7-63e7cd13e9ab" containerName="payload-processing"
Sep 08 16:11:21.481537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481425    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="383040d6-75c0-4de7-b4a7-63e7cd13e9ab" containerName="payload-processing"
Sep 08 16:11:21.481537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481486    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d44de2e-28e7-45c5-b2a6-c56d71d01f5b" containerName="payload-pre-processing"
Sep 08 16:11:21.481537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481494    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="383040d6-75c0-4de7-b4a7-63e7cd13e9ab" containerName="payload-processing"
Sep 08 16:11:21.481537 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.481503    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e90ae7c-33d9-4526-a930-b716ffae7113" containerName="revoke-keys"
Sep 08 16:11:21.484700 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.484683    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"
Sep 08 16:11:21.486758 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.486739    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:11:21.486847 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.486807    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:11:21.492289 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.492235    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"]
Sep 08 16:11:21.608863 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.608834    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f33aa6-80e8-478f-b9eb-f753c212396a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d\" (UID: \"b9f33aa6-80e8-478f-b9eb-f753c212396a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"
Sep 08 16:11:21.709796 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.709764    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f33aa6-80e8-478f-b9eb-f753c212396a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d\" (UID: \"b9f33aa6-80e8-478f-b9eb-f753c212396a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"
Sep 08 16:11:21.710375 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.710354    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f33aa6-80e8-478f-b9eb-f753c212396a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d\" (UID: \"b9f33aa6-80e8-478f-b9eb-f753c212396a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"
Sep 08 16:11:21.797861 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:21.797773    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"
Sep 08 16:11:22.071757 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:22.071728    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"]
Sep 08 16:11:22.073604 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:11:22.073576    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f33aa6_80e8_478f_b9eb_f753c212396a.slice/crio-ff9c7c43e629605583409c0a557d483bce5550da57d71d37788133d4638689d0 WatchSource:0}: Error finding container ff9c7c43e629605583409c0a557d483bce5550da57d71d37788133d4638689d0: Status 404 returned error can't find the container with id ff9c7c43e629605583409c0a557d483bce5550da57d71d37788133d4638689d0
Sep 08 16:11:22.540356 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:22.540323    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d" event={"ID":"b9f33aa6-80e8-478f-b9eb-f753c212396a","Type":"ContainerStarted","Data":"52587e9d8cc23ce62443a0a5d51ecbb3ee5e9c9b7f79fb43945e5f09a83ef6ef"}
Sep 08 16:11:22.540736 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:22.540404    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d" event={"ID":"b9f33aa6-80e8-478f-b9eb-f753c212396a","Type":"ContainerStarted","Data":"ff9c7c43e629605583409c0a557d483bce5550da57d71d37788133d4638689d0"}
Sep 08 16:11:22.557289 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:22.557240    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d" podStartSLOduration=1.557226024 podStartE2EDuration="1.557226024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:11:22.555516845 +0000 UTC m=+1592.051924813" watchObservedRunningTime="2026-09-08 16:11:22.557226024 +0000 UTC m=+1592.053633991"
Sep 08 16:11:23.547803 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:23.547767    2552 generic.go:356] "Generic (PLEG): container finished" podID="b9f33aa6-80e8-478f-b9eb-f753c212396a" containerID="52587e9d8cc23ce62443a0a5d51ecbb3ee5e9c9b7f79fb43945e5f09a83ef6ef" exitCode=0
Sep 08 16:11:23.548275 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:23.547857    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d" event={"ID":"b9f33aa6-80e8-478f-b9eb-f753c212396a","Type":"ContainerDied","Data":"52587e9d8cc23ce62443a0a5d51ecbb3ee5e9c9b7f79fb43945e5f09a83ef6ef"}
Sep 08 16:11:24.683843 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:24.683812    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"
Sep 08 16:11:24.836114 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:24.836014    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f33aa6-80e8-478f-b9eb-f753c212396a-maas-api-service-ca\") pod \"b9f33aa6-80e8-478f-b9eb-f753c212396a\" (UID: \"b9f33aa6-80e8-478f-b9eb-f753c212396a\") "
Sep 08 16:11:24.836409 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:24.836384    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9f33aa6-80e8-478f-b9eb-f753c212396a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9f33aa6-80e8-478f-b9eb-f753c212396a" (UID: "b9f33aa6-80e8-478f-b9eb-f753c212396a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:11:24.937667 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:24.937640    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9f33aa6-80e8-478f-b9eb-f753c212396a-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:25.560371 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:25.560342    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"
Sep 08 16:11:25.560371 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:25.560360    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d" event={"ID":"b9f33aa6-80e8-478f-b9eb-f753c212396a","Type":"ContainerDied","Data":"ff9c7c43e629605583409c0a557d483bce5550da57d71d37788133d4638689d0"}
Sep 08 16:11:25.560624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:25.560406    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff9c7c43e629605583409c0a557d483bce5550da57d71d37788133d4638689d0"
Sep 08 16:11:26.365683 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.365658    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:11:26.453322 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453238    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istiod-ca-cert\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453322 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453273    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-podinfo\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453322 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453294    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgz2x\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-kube-api-access-bgz2x\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453322 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453323    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-certs\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453668 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453521    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-socket\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453668 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453612    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-token\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453668 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453615    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:11:26.453839 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453705    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:11:26.453839 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453722    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-envoy\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453839 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453749    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:11:26.453839 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453759    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-data\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.453839 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453822    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-credential-socket\") pod \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\" (UID: \"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40\") "
Sep 08 16:11:26.454139 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.453951    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:11:26.454139 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.454132    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:11:26.454258 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.454203    2552 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-data\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.454258 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.454223    2552 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-credential-socket\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.454258 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.454236    2552 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istiod-ca-cert\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.454258 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.454250    2552 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.454525 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.454261    2552 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-workload-socket\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.455624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.455600    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-kube-api-access-bgz2x" (OuterVolumeSpecName: "kube-api-access-bgz2x") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "kube-api-access-bgz2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:11:26.455778 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.455718    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 16:11:26.456052 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.456012    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:11:26.456052 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.456037    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" (UID: "bd0ff080-ed99-4df2-b82b-eb57ff7dcf40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:11:26.555432 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.555400    2552 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-podinfo\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.555432 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.555429    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgz2x\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-kube-api-access-bgz2x\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.555432 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.555438    2552 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-token\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.555667 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.555447    2552 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40-istio-envoy\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:26.568586 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.568555    2552 generic.go:356] "Generic (PLEG): container finished" podID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerID="56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872" exitCode=0
Sep 08 16:11:26.568723 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.568618    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"
Sep 08 16:11:26.568723 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.568637    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" event={"ID":"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40","Type":"ContainerDied","Data":"56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872"}
Sep 08 16:11:26.568723 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.568695    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb" event={"ID":"bd0ff080-ed99-4df2-b82b-eb57ff7dcf40","Type":"ContainerDied","Data":"c1d99a63c6edcda5397337153b6f0d55fc054c1c2024f5e51dc74112d7df588c"}
Sep 08 16:11:26.568723 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.568719    2552 scope.go:117] "RemoveContainer" containerID="56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872"
Sep 08 16:11:26.581493 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.581476    2552 scope.go:117] "RemoveContainer" containerID="1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d"
Sep 08 16:11:26.589826 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.589810    2552 scope.go:117] "RemoveContainer" containerID="56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872"
Sep 08 16:11:26.590115 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:11:26.590096    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872\": container with ID starting with 56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872 not found: ID does not exist" containerID="56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872"
Sep 08 16:11:26.590181 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.590122    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872"} err="failed to get container status \"56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872\": rpc error: code = NotFound desc = could not find container \"56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872\": container with ID starting with 56c8c859a4920cad6004e310c7e3e7a72f24e3e25b4e8d70c00022efd8adf872 not found: ID does not exist"
Sep 08 16:11:26.590181 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.590140    2552 scope.go:117] "RemoveContainer" containerID="1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d"
Sep 08 16:11:26.590405 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:11:26.590387    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d\": container with ID starting with 1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d not found: ID does not exist" containerID="1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d"
Sep 08 16:11:26.590447 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.590410    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d"} err="failed to get container status \"1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d\": rpc error: code = NotFound desc = could not find container \"1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d\": container with ID starting with 1df7a34bda618c1296e3afab0e7fa2df68ae09b590222782ad47eeb25809007d not found: ID does not exist"
Sep 08 16:11:26.597827 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.597804    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"]
Sep 08 16:11:26.601389 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:26.601368    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-cf851c-openshift-default-59d97d7d7d-q7kgb"]
Sep 08 16:11:27.101062 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:27.101017    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" path="/var/lib/kubelet/pods/bd0ff080-ed99-4df2-b82b-eb57ff7dcf40/volumes"
Sep 08 16:11:32.064178 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:32.064141    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"]
Sep 08 16:11:32.066901 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:32.066871    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7a7297-vl68d"]
Sep 08 16:11:33.102112 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:33.102080    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9f33aa6-80e8-478f-b9eb-f753c212396a" path="/var/lib/kubelet/pods/b9f33aa6-80e8-478f-b9eb-f753c212396a/volumes"
Sep 08 16:11:38.920988 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.920951    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"]
Sep 08 16:11:38.921463 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921447    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy"
Sep 08 16:11:38.921463 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921464    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy"
Sep 08 16:11:38.921538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921487    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9f33aa6-80e8-478f-b9eb-f753c212396a" containerName="revoke-keys"
Sep 08 16:11:38.921538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921495    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9f33aa6-80e8-478f-b9eb-f753c212396a" containerName="revoke-keys"
Sep 08 16:11:38.921538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921506    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy"
Sep 08 16:11:38.921538 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921512    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy"
Sep 08 16:11:38.921659 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921568    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy"
Sep 08 16:11:38.921659 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921576    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9f33aa6-80e8-478f-b9eb-f753c212396a" containerName="revoke-keys"
Sep 08 16:11:38.921659 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.921582    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0ff080-ed99-4df2-b82b-eb57ff7dcf40" containerName="istio-proxy"
Sep 08 16:11:38.926012 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.925990    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"
Sep 08 16:11:38.928082 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.928063    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65h2p\""
Sep 08 16:11:38.928202 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.928082    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 16:11:38.935074 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.934574    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"]
Sep 08 16:11:38.969708 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:38.969670    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d99d77-6dd4-46d5-9112-673be8e7f2e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j\" (UID: \"c8d99d77-6dd4-46d5-9112-673be8e7f2e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"
Sep 08 16:11:39.070610 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:39.070577    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d99d77-6dd4-46d5-9112-673be8e7f2e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j\" (UID: \"c8d99d77-6dd4-46d5-9112-673be8e7f2e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"
Sep 08 16:11:39.071216 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:39.071199    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d99d77-6dd4-46d5-9112-673be8e7f2e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j\" (UID: \"c8d99d77-6dd4-46d5-9112-673be8e7f2e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"
Sep 08 16:11:39.238819 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:39.238714    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"
Sep 08 16:11:39.507614 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:39.507530    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"]
Sep 08 16:11:39.511601 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:11:39.511561    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d99d77_6dd4_46d5_9112_673be8e7f2e3.slice/crio-c13a3668cdde16ae2377dfad1c7b14ee8e187f64c975f38de621652f108e1c9c WatchSource:0}: Error finding container c13a3668cdde16ae2377dfad1c7b14ee8e187f64c975f38de621652f108e1c9c: Status 404 returned error can't find the container with id c13a3668cdde16ae2377dfad1c7b14ee8e187f64c975f38de621652f108e1c9c
Sep 08 16:11:39.650613 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:39.650580    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j" event={"ID":"c8d99d77-6dd4-46d5-9112-673be8e7f2e3","Type":"ContainerStarted","Data":"c13a3668cdde16ae2377dfad1c7b14ee8e187f64c975f38de621652f108e1c9c"}
Sep 08 16:11:40.657997 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:40.657959    2552 generic.go:356] "Generic (PLEG): container finished" podID="c8d99d77-6dd4-46d5-9112-673be8e7f2e3" containerID="85f7784612c638d6ebc491c5f133592033e924cb461f1ffee16d055a91365728" exitCode=0
Sep 08 16:11:40.658382 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:40.658058    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j" event={"ID":"c8d99d77-6dd4-46d5-9112-673be8e7f2e3","Type":"ContainerDied","Data":"85f7784612c638d6ebc491c5f133592033e924cb461f1ffee16d055a91365728"}
Sep 08 16:11:41.796125 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:41.796100    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"
Sep 08 16:11:41.893912 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:41.893880    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d99d77-6dd4-46d5-9112-673be8e7f2e3-maas-api-service-ca\") pod \"c8d99d77-6dd4-46d5-9112-673be8e7f2e3\" (UID: \"c8d99d77-6dd4-46d5-9112-673be8e7f2e3\") "
Sep 08 16:11:41.894256 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:41.894230    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8d99d77-6dd4-46d5-9112-673be8e7f2e3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8d99d77-6dd4-46d5-9112-673be8e7f2e3" (UID: "c8d99d77-6dd4-46d5-9112-673be8e7f2e3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:11:41.994705 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:41.994624    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d99d77-6dd4-46d5-9112-673be8e7f2e3-maas-api-service-ca\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:11:42.670237 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:42.670209    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"
Sep 08 16:11:42.670237 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:42.670228    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j" event={"ID":"c8d99d77-6dd4-46d5-9112-673be8e7f2e3","Type":"ContainerDied","Data":"c13a3668cdde16ae2377dfad1c7b14ee8e187f64c975f38de621652f108e1c9c"}
Sep 08 16:11:42.670437 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:42.670271    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c13a3668cdde16ae2377dfad1c7b14ee8e187f64c975f38de621652f108e1c9c"
Sep 08 16:11:49.509075 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:49.508989    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"]
Sep 08 16:11:49.511568 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:49.511539    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-108124-5l84j"]
Sep 08 16:11:51.101998 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:11:51.101965    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8d99d77-6dd4-46d5-9112-673be8e7f2e3" path="/var/lib/kubelet/pods/c8d99d77-6dd4-46d5-9112-673be8e7f2e3/volumes"
Sep 08 16:12:16.437701 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.437666    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75db7f7669-9w88x"]
Sep 08 16:12:16.438343 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.437909    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75db7f7669-9w88x" podUID="9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" containerName="manager" containerID="cri-o://f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d" gracePeriod=30
Sep 08 16:12:16.688231 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.688174    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 16:12:16.726059 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.726007    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-metrics-certs\") pod \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") "
Sep 08 16:12:16.726201 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.726091    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-webhook-cert\") pod \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") "
Sep 08 16:12:16.726201 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.726133    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45fpk\" (UniqueName: \"kubernetes.io/projected/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-kube-api-access-45fpk\") pod \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\" (UID: \"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63\") "
Sep 08 16:12:16.728460 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.728425    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-kube-api-access-45fpk" (OuterVolumeSpecName: "kube-api-access-45fpk") pod "9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" (UID: "9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63"). InnerVolumeSpecName "kube-api-access-45fpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:12:16.728696 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.728667    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" (UID: "9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:12:16.728896 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.728868    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" (UID: "9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 16:12:16.827285 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.827250    2552 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-metrics-certs\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:12:16.827285 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.827280    2552 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-webhook-cert\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:12:16.827285 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.827291    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45fpk\" (UniqueName: \"kubernetes.io/projected/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63-kube-api-access-45fpk\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:12:16.854365 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.854332    2552 generic.go:356] "Generic (PLEG): container finished" podID="9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" containerID="f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d" exitCode=0
Sep 08 16:12:16.854509 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.854378    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75db7f7669-9w88x"
Sep 08 16:12:16.854509 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.854408    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75db7f7669-9w88x" event={"ID":"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63","Type":"ContainerDied","Data":"f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d"}
Sep 08 16:12:16.854509 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.854469    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75db7f7669-9w88x" event={"ID":"9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63","Type":"ContainerDied","Data":"9f4004915be2d8b877500c7e7ae8bd6af8cf0f21a4aedb154d6dc5ebe6926464"}
Sep 08 16:12:16.854509 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.854493    2552 scope.go:117] "RemoveContainer" containerID="f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d"
Sep 08 16:12:16.864354 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.864332    2552 scope.go:117] "RemoveContainer" containerID="f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d"
Sep 08 16:12:16.864635 ip-10-0-136-224 kubenswrapper[2552]: E0908 16:12:16.864613    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d\": container with ID starting with f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d not found: ID does not exist" containerID="f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d"
Sep 08 16:12:16.864703 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.864646    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d"} err="failed to get container status \"f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d\": rpc error: code = NotFound desc = could not find container \"f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d\": container with ID starting with f6a13afe851da082ed7b9b5978a0dc02e183328fbf4886ec7c84df19b8e0cc1d not found: ID does not exist"
Sep 08 16:12:16.874855 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.874834    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75db7f7669-9w88x"]
Sep 08 16:12:16.878338 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:16.878317    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75db7f7669-9w88x"]
Sep 08 16:12:17.104258 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:12:17.104162    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" path="/var/lib/kubelet/pods/9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63/volumes"
Sep 08 16:13:51.329998 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:13:51.329964    2552 scope.go:117] "RemoveContainer" containerID="47779ef24e47fbb2bb83040c392d567b793e4c6d8fca6bf5a6a560baf5888dd1"
Sep 08 16:13:51.338362 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:13:51.338342    2552 scope.go:117] "RemoveContainer" containerID="934322789d31144a7e06bac3b2e970ccc36870918ed832355a7817eecdc9f611"
Sep 08 16:13:51.349005 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:13:51.347774    2552 scope.go:117] "RemoveContainer" containerID="f9096217da119f80df6cab420d534e8cdf65db84f516a3f2e4f862bb7fd7f056"
Sep 08 16:14:46.844785 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:46.844687    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"]
Sep 08 16:14:46.845372 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:46.844892    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc" podUID="2e4d6656-58a3-4327-ae63-fa4a0165f5b8" containerName="manager" containerID="cri-o://b3bd9a4a07a94a35950996f658b453fbb6a1762cc18ffdef5de66dbfed5e2734" gracePeriod=10
Sep 08 16:14:48.664073 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.664019    2552 generic.go:356] "Generic (PLEG): container finished" podID="2e4d6656-58a3-4327-ae63-fa4a0165f5b8" containerID="b3bd9a4a07a94a35950996f658b453fbb6a1762cc18ffdef5de66dbfed5e2734" exitCode=0
Sep 08 16:14:48.664073 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.664063    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc" event={"ID":"2e4d6656-58a3-4327-ae63-fa4a0165f5b8","Type":"ContainerDied","Data":"b3bd9a4a07a94a35950996f658b453fbb6a1762cc18ffdef5de66dbfed5e2734"}
Sep 08 16:14:48.791819 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.791791    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 16:14:48.880902 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.880812    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-extensions-socket-volume\") pod \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") "
Sep 08 16:14:48.880902 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.880861    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvm6x\" (UniqueName: \"kubernetes.io/projected/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-kube-api-access-jvm6x\") pod \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\" (UID: \"2e4d6656-58a3-4327-ae63-fa4a0165f5b8\") "
Sep 08 16:14:48.881282 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.881254    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2e4d6656-58a3-4327-ae63-fa4a0165f5b8" (UID: "2e4d6656-58a3-4327-ae63-fa4a0165f5b8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:14:48.882924 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.882902    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-kube-api-access-jvm6x" (OuterVolumeSpecName: "kube-api-access-jvm6x") pod "2e4d6656-58a3-4327-ae63-fa4a0165f5b8" (UID: "2e4d6656-58a3-4327-ae63-fa4a0165f5b8"). InnerVolumeSpecName "kube-api-access-jvm6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:14:48.981665 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.981631    2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-extensions-socket-volume\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:14:48.981665 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:48.981658    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvm6x\" (UniqueName: \"kubernetes.io/projected/2e4d6656-58a3-4327-ae63-fa4a0165f5b8-kube-api-access-jvm6x\") on node \"ip-10-0-136-224.ec2.internal\" DevicePath \"\""
Sep 08 16:14:49.670790 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:49.670762    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"
Sep 08 16:14:49.671214 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:49.670763    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc" event={"ID":"2e4d6656-58a3-4327-ae63-fa4a0165f5b8","Type":"ContainerDied","Data":"97141ce5f6993b195194310925dc1fc5e6a220421e350d2e2656054e968730d1"}
Sep 08 16:14:49.671214 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:49.670890    2552 scope.go:117] "RemoveContainer" containerID="b3bd9a4a07a94a35950996f658b453fbb6a1762cc18ffdef5de66dbfed5e2734"
Sep 08 16:14:49.686712 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:49.686687    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"]
Sep 08 16:14:49.690161 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:49.690138    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8khvc"]
Sep 08 16:14:51.102385 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:51.102352    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e4d6656-58a3-4327-ae63-fa4a0165f5b8" path="/var/lib/kubelet/pods/2e4d6656-58a3-4327-ae63-fa4a0165f5b8/volumes"
Sep 08 16:14:51.374130 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:14:51.374100    2552 scope.go:117] "RemoveContainer" containerID="06d2d90b40cbb6e02648ac8c9f51a93352c141b534aba29cf828479ee7d72c8c"
Sep 08 16:15:47.595442 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595398    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"]
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595797    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8d99d77-6dd4-46d5-9112-673be8e7f2e3" containerName="revoke-keys"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595810    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8d99d77-6dd4-46d5-9112-673be8e7f2e3" containerName="revoke-keys"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595835    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e4d6656-58a3-4327-ae63-fa4a0165f5b8" containerName="manager"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595841    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e4d6656-58a3-4327-ae63-fa4a0165f5b8" containerName="manager"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595849    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" containerName="manager"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595858    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" containerName="manager"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595921    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e4d6656-58a3-4327-ae63-fa4a0165f5b8" containerName="manager"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595932    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8d99d77-6dd4-46d5-9112-673be8e7f2e3" containerName="revoke-keys"
Sep 08 16:15:47.596049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.595942    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9424134d-5bcd-44ff-bbd8-0e6c7ef9cd63" containerName="manager"
Sep 08 16:15:47.598986 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.598966    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:47.605362 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.605341    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t8mht\""
Sep 08 16:15:47.631206 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.631181    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"]
Sep 08 16:15:47.756498 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.756455    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8dll\" (UniqueName: \"kubernetes.io/projected/74d4326e-675d-4cea-8101-0060013d460e-kube-api-access-x8dll\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bfcsg\" (UID: \"74d4326e-675d-4cea-8101-0060013d460e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:47.756680 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.756509    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d4326e-675d-4cea-8101-0060013d460e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bfcsg\" (UID: \"74d4326e-675d-4cea-8101-0060013d460e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:47.857519 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.857428    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8dll\" (UniqueName: \"kubernetes.io/projected/74d4326e-675d-4cea-8101-0060013d460e-kube-api-access-x8dll\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bfcsg\" (UID: \"74d4326e-675d-4cea-8101-0060013d460e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:47.857519 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.857476    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d4326e-675d-4cea-8101-0060013d460e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bfcsg\" (UID: \"74d4326e-675d-4cea-8101-0060013d460e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:47.857861 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.857841    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d4326e-675d-4cea-8101-0060013d460e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bfcsg\" (UID: \"74d4326e-675d-4cea-8101-0060013d460e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:47.865733 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.865709    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8dll\" (UniqueName: \"kubernetes.io/projected/74d4326e-675d-4cea-8101-0060013d460e-kube-api-access-x8dll\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bfcsg\" (UID: \"74d4326e-675d-4cea-8101-0060013d460e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:47.909634 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:47.909607    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:48.190266 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:48.190240    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"]
Sep 08 16:15:48.192430 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:15:48.192402    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d4326e_675d_4cea_8101_0060013d460e.slice/crio-c748321add63cf200d55bc24965ea481005e511f01358f691afe59417f417df3 WatchSource:0}: Error finding container c748321add63cf200d55bc24965ea481005e511f01358f691afe59417f417df3: Status 404 returned error can't find the container with id c748321add63cf200d55bc24965ea481005e511f01358f691afe59417f417df3
Sep 08 16:15:48.970104 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:48.970067    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg" event={"ID":"74d4326e-675d-4cea-8101-0060013d460e","Type":"ContainerStarted","Data":"dac1398a5e3c5983213f37740288abbab0f14dedd128c72f781c7eccbdb1da07"}
Sep 08 16:15:48.970476 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:48.970132    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:15:48.970476 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:48.970150    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg" event={"ID":"74d4326e-675d-4cea-8101-0060013d460e","Type":"ContainerStarted","Data":"c748321add63cf200d55bc24965ea481005e511f01358f691afe59417f417df3"}
Sep 08 16:15:48.996611 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:48.996560    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg" podStartSLOduration=1.996545241 podStartE2EDuration="1.996545241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:15:48.99344108 +0000 UTC m=+1858.489849046" watchObservedRunningTime="2026-09-08 16:15:48.996545241 +0000 UTC m=+1858.492953207"
Sep 08 16:15:59.977971 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:15:59.977935    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bfcsg"
Sep 08 16:16:51.403349 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:16:51.403317    2552 scope.go:117] "RemoveContainer" containerID="d723732111615e452bed41c258fff4066169539067de35dd10e97c3e4b4e67c8"
Sep 08 16:16:51.411541 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:16:51.411513    2552 scope.go:117] "RemoveContainer" containerID="f929730c509f2fb78a178c653452f9ebd81199315c4b850b086d6ed2ce58ec53"
Sep 08 16:17:51.434431 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:17:51.434345    2552 scope.go:117] "RemoveContainer" containerID="85f7784612c638d6ebc491c5f133592033e924cb461f1ffee16d055a91365728"
Sep 08 16:17:51.442793 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:17:51.442770    2552 scope.go:117] "RemoveContainer" containerID="52587e9d8cc23ce62443a0a5d51ecbb3ee5e9c9b7f79fb43945e5f09a83ef6ef"
Sep 08 16:18:02.231795 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:02.231768    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nrzlr_8b038f1d-af2d-4c8f-990a-019779b860bc/server/0.log"
Sep 08 16:18:02.350295 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:02.350258    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hlpsc_992f3f5a-d514-441a-b69c-d44a7d9d2311/manager/0.log"
Sep 08 16:18:02.577967 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:02.577886    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d4b5866d6-jfm4l_6bcd7edd-335e-4b49-b550-044bc1047133/copy-manifests/0.log"
Sep 08 16:18:02.596356 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:02.596327    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d4b5866d6-jfm4l_6bcd7edd-335e-4b49-b550-044bc1047133/manager/0.log"
Sep 08 16:18:03.272588 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:03.272562    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814720-qqspg_93653ae7-bf90-40ee-8b3b-4ef501943008/cleanup/0.log"
Sep 08 16:18:04.471339 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:04.471309    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ftwxm_ada175f5-1375-48d0-93ee-9ffc7e8c2719/manager/0.log"
Sep 08 16:18:04.578187 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:04.578158    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-cdkr6_1eb678b0-f5b9-437a-8388-fb6b5b6d20e2/kuadrant-console-plugin/0.log"
Sep 08 16:18:04.702106 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:04.702074    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-76f8f686bf-bfcsg_74d4326e-675d-4cea-8101-0060013d460e/manager/0.log"
Sep 08 16:18:05.287236 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:05.287207    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh_be0b2394-e834-406c-a95f-c06bd741594a/istio-proxy/0.log"
Sep 08 16:18:05.518531 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:05.518506    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74bfc7b5fc-725tn_0e594c7f-af6f-432a-aec2-fd4399dbd462/kube-auth-proxy/0.log"
Sep 08 16:18:05.762610 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:05.762576    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ksf69_e8c13238-ca5b-4de2-a21f-4f187ca413b6/istio-proxy/1.log"
Sep 08 16:18:06.013847 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:06.013764    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9g54m_26b382fb-6099-4697-91f1-c4d30851a597/payload-processing/0.log"
Sep 08 16:18:06.580098 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:06.580060    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-cdrxh_b1a009d8-3957-48e6-ae8e-5a2a3c92ef6d/main/0.log"
Sep 08 16:18:06.934422 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:06.934386    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6v48f_f6dc8cd4-9eb8-445d-b3b6-5dbf80a2e44d/main/0.log"
Sep 08 16:18:07.160837 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:07.160808    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-k9tnk_20f81e62-734b-4fe0-8903-fd0775d5e750/main/0.log"
Sep 08 16:18:15.992869 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:15.992831    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wqb2w_80e81107-5b70-4123-87bf-8c0827d277d2/global-pull-secret-syncer/0.log"
Sep 08 16:18:16.137152 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:16.137115    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xd4bw_dfb6f525-8a24-40f3-841a-6798d886a200/konnectivity-agent/0.log"
Sep 08 16:18:16.179624 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:16.179597    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-224.ec2.internal_96fc930a9b55d927d19f464231ac2a2b/haproxy/0.log"
Sep 08 16:18:20.888545 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:20.888514    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ftwxm_ada175f5-1375-48d0-93ee-9ffc7e8c2719/manager/0.log"
Sep 08 16:18:20.914252 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:20.914228    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-cdkr6_1eb678b0-f5b9-437a-8388-fb6b5b6d20e2/kuadrant-console-plugin/0.log"
Sep 08 16:18:20.970671 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:20.970639    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-76f8f686bf-bfcsg_74d4326e-675d-4cea-8101-0060013d460e/manager/0.log"
Sep 08 16:18:22.846090 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:22.845983    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8l6qg_56fb494c-88ee-4cd2-bb69-93e1d5e5e507/kube-state-metrics/0.log"
Sep 08 16:18:22.864901 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:22.864874    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8l6qg_56fb494c-88ee-4cd2-bb69-93e1d5e5e507/kube-rbac-proxy-main/0.log"
Sep 08 16:18:22.884349 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:22.884298    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8l6qg_56fb494c-88ee-4cd2-bb69-93e1d5e5e507/kube-rbac-proxy-self/0.log"
Sep 08 16:18:22.921563 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:22.921529    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5487ccc788-sw5tv_ff809900-2d1e-405a-be92-f1233a4daa58/metrics-server/0.log"
Sep 08 16:18:23.036837 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.036805    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6bkmd_4ae14a40-5818-48f5-a879-1e7b8520ced1/node-exporter/0.log"
Sep 08 16:18:23.056137 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.056114    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6bkmd_4ae14a40-5818-48f5-a879-1e7b8520ced1/kube-rbac-proxy/0.log"
Sep 08 16:18:23.073824 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.073806    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6bkmd_4ae14a40-5818-48f5-a879-1e7b8520ced1/init-textfile/0.log"
Sep 08 16:18:23.184311 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.184282    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-nt7hb_bbe255b2-94df-4704-b7d2-91ca9c419a28/kube-rbac-proxy-main/0.log"
Sep 08 16:18:23.206634 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.206607    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-nt7hb_bbe255b2-94df-4704-b7d2-91ca9c419a28/kube-rbac-proxy-self/0.log"
Sep 08 16:18:23.227600 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.227574    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-nt7hb_bbe255b2-94df-4704-b7d2-91ca9c419a28/openshift-state-metrics/0.log"
Sep 08 16:18:23.281794 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.281759    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a/prometheus/0.log"
Sep 08 16:18:23.299909 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.299882    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a/config-reloader/0.log"
Sep 08 16:18:23.332884 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.332853    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a/thanos-sidecar/0.log"
Sep 08 16:18:23.351619 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.351600    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a/kube-rbac-proxy-web/0.log"
Sep 08 16:18:23.371079 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.371058    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a/kube-rbac-proxy/0.log"
Sep 08 16:18:23.389634 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.389609    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a/kube-rbac-proxy-thanos/0.log"
Sep 08 16:18:23.407924 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.407899    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9eba29fc-a5b7-4ef3-9fb7-a77b43fa868a/init-config-reloader/0.log"
Sep 08 16:18:23.436074 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.435991    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-frxpq_afc3611e-8b39-4918-99ed-8121d46112d6/prometheus-operator/0.log"
Sep 08 16:18:23.454660 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.454640    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-frxpq_afc3611e-8b39-4918-99ed-8121d46112d6/kube-rbac-proxy/0.log"
Sep 08 16:18:23.607702 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.607679    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4df7cffb-ql774_62e0dc58-e0f1-4071-baca-5fa054f1bd78/thanos-query/0.log"
Sep 08 16:18:23.629245 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.629221    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4df7cffb-ql774_62e0dc58-e0f1-4071-baca-5fa054f1bd78/kube-rbac-proxy-web/0.log"
Sep 08 16:18:23.649109 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.649092    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4df7cffb-ql774_62e0dc58-e0f1-4071-baca-5fa054f1bd78/kube-rbac-proxy/0.log"
Sep 08 16:18:23.671158 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.671140    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4df7cffb-ql774_62e0dc58-e0f1-4071-baca-5fa054f1bd78/prom-label-proxy/0.log"
Sep 08 16:18:23.692547 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.692490    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4df7cffb-ql774_62e0dc58-e0f1-4071-baca-5fa054f1bd78/kube-rbac-proxy-rules/0.log"
Sep 08 16:18:23.716903 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:23.716875    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b4df7cffb-ql774_62e0dc58-e0f1-4071-baca-5fa054f1bd78/kube-rbac-proxy-metrics/0.log"
Sep 08 16:18:26.616174 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.616138    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"]
Sep 08 16:18:26.619635 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.619614    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.621715 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.621695    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jllsk\"/\"kube-root-ca.crt\""
Sep 08 16:18:26.622420 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.622404    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jllsk\"/\"openshift-service-ca.crt\""
Sep 08 16:18:26.622507 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.622490    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jllsk\"/\"default-dockercfg-m9wgw\""
Sep 08 16:18:26.629363 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.629342    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"]
Sep 08 16:18:26.700100 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.700069    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2snwm\" (UniqueName: \"kubernetes.io/projected/de00d639-6707-4723-9459-ab0fe9aab4ed-kube-api-access-2snwm\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.700100 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.700100    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-proc\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.700281 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.700121    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-sys\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.700281 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.700143    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-lib-modules\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.700281 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.700196    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-podres\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801581 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801546    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-lib-modules\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801581 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801584    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-podres\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801781 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801664    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2snwm\" (UniqueName: \"kubernetes.io/projected/de00d639-6707-4723-9459-ab0fe9aab4ed-kube-api-access-2snwm\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801781 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801687    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-proc\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801781 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801710    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-sys\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801781 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801728    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-lib-modules\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801912 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801779    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-proc\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801912 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801792    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-podres\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.801912 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.801787    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de00d639-6707-4723-9459-ab0fe9aab4ed-sys\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.809562 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.809528    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2snwm\" (UniqueName: \"kubernetes.io/projected/de00d639-6707-4723-9459-ab0fe9aab4ed-kube-api-access-2snwm\") pod \"perf-node-gather-daemonset-r8s4x\" (UID: \"de00d639-6707-4723-9459-ab0fe9aab4ed\") " pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:26.930238 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:26.930140    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:27.201805 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:27.201779    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"]
Sep 08 16:18:27.204186 ip-10-0-136-224 kubenswrapper[2552]: W0908 16:18:27.204156    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podde00d639_6707_4723_9459_ab0fe9aab4ed.slice/crio-c1a3534a50a35f481b56c7e0e5c13b734190cf606632e555be7013e94f033b5d WatchSource:0}: Error finding container c1a3534a50a35f481b56c7e0e5c13b734190cf606632e555be7013e94f033b5d: Status 404 returned error can't find the container with id c1a3534a50a35f481b56c7e0e5c13b734190cf606632e555be7013e94f033b5d
Sep 08 16:18:27.780485 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:27.780447    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x" event={"ID":"de00d639-6707-4723-9459-ab0fe9aab4ed","Type":"ContainerStarted","Data":"858a6046f3efc8233d1be86bbc6e75e743b3a260e03363efc4d56d223868bf10"}
Sep 08 16:18:27.780876 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:27.780503    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x" event={"ID":"de00d639-6707-4723-9459-ab0fe9aab4ed","Type":"ContainerStarted","Data":"c1a3534a50a35f481b56c7e0e5c13b734190cf606632e555be7013e94f033b5d"}
Sep 08 16:18:27.780876 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:27.780543    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:27.795947 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:27.795899    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x" podStartSLOduration=1.7958852379999999 podStartE2EDuration="1.795885238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:18:27.793939801 +0000 UTC m=+2017.290347805" watchObservedRunningTime="2026-09-08 16:18:27.795885238 +0000 UTC m=+2017.292293204"
Sep 08 16:18:28.049383 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:28.049277    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k9xkq_bddd1262-8293-4b27-ba17-153260e2cb4a/dns/0.log"
Sep 08 16:18:28.066479 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:28.066452    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k9xkq_bddd1262-8293-4b27-ba17-153260e2cb4a/kube-rbac-proxy/0.log"
Sep 08 16:18:28.104925 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:28.104901    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dkswd_224be9e3-b75d-4315-b6d0-370cf1b6babd/dns-node-resolver/0.log"
Sep 08 16:18:28.619424 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:28.619396    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kltj9_4a606414-2982-4f7c-9cfa-0fbc3e7d84ad/node-ca/0.log"
Sep 08 16:18:29.535574 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:29.535541    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh_be0b2394-e834-406c-a95f-c06bd741594a/istio-proxy/0.log"
Sep 08 16:18:29.789089 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:29.788990    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74bfc7b5fc-725tn_0e594c7f-af6f-432a-aec2-fd4399dbd462/kube-auth-proxy/0.log"
Sep 08 16:18:29.838225 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:29.838192    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ksf69_e8c13238-ca5b-4de2-a21f-4f187ca413b6/istio-proxy/0.log"
Sep 08 16:18:29.853575 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:29.853541    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ksf69_e8c13238-ca5b-4de2-a21f-4f187ca413b6/istio-proxy/1.log"
Sep 08 16:18:29.936737 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:29.936705    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9g54m_26b382fb-6099-4697-91f1-c4d30851a597/payload-processing/0.log"
Sep 08 16:18:30.808295 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:30.808270    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxr6z_06ae3d7a-f203-4581-bb84-4f732af55a4a/serve-healthcheck-canary/0.log"
Sep 08 16:18:31.412544 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:31.412513    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jbvz5_1e3ac920-58a2-4334-9794-0e0a05753954/kube-rbac-proxy/0.log"
Sep 08 16:18:31.433259 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:31.433236    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jbvz5_1e3ac920-58a2-4334-9794-0e0a05753954/exporter/0.log"
Sep 08 16:18:31.452297 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:31.452272    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jbvz5_1e3ac920-58a2-4334-9794-0e0a05753954/extractor/0.log"
Sep 08 16:18:33.670724 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:33.670691    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nrzlr_8b038f1d-af2d-4c8f-990a-019779b860bc/server/0.log"
Sep 08 16:18:33.795934 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:33.795897    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hlpsc_992f3f5a-d514-441a-b69c-d44a7d9d2311/manager/0.log"
Sep 08 16:18:33.798008 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:33.797984    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jllsk/perf-node-gather-daemonset-r8s4x"
Sep 08 16:18:33.964831 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:33.964746    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d4b5866d6-jfm4l_6bcd7edd-335e-4b49-b550-044bc1047133/manager/0.log"
Sep 08 16:18:33.979414 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:33.979388    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d4b5866d6-jfm4l_6bcd7edd-335e-4b49-b550-044bc1047133/copy-manifests/0.log"
Sep 08 16:18:35.199418 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:35.199388    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-578d6cf975-lwzlv_538aa371-7e65-4262-81e2-05b425b4c310/manager/0.log"
Sep 08 16:18:35.220306 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:35.220283    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-927zv_cd85a285-7ac6-4927-aa5e-b98535132f6d/openshift-lws-operator/0.log"
Sep 08 16:18:35.867912 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:35.867878    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh_be0b2394-e834-406c-a95f-c06bd741594a/istio-proxy/0.log"
Sep 08 16:18:36.122668 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:36.122581    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74bfc7b5fc-725tn_0e594c7f-af6f-432a-aec2-fd4399dbd462/kube-auth-proxy/0.log"
Sep 08 16:18:36.162407 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:36.162372    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ksf69_e8c13238-ca5b-4de2-a21f-4f187ca413b6/istio-proxy/0.log"
Sep 08 16:18:36.179263 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:36.179234    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ksf69_e8c13238-ca5b-4de2-a21f-4f187ca413b6/istio-proxy/1.log"
Sep 08 16:18:36.249777 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:36.249750    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9g54m_26b382fb-6099-4697-91f1-c4d30851a597/payload-processing/0.log"
Sep 08 16:18:37.805191 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:37.805165    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-9mmzh_be0b2394-e834-406c-a95f-c06bd741594a/istio-proxy/0.log"
Sep 08 16:18:38.045592 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:38.045565    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74bfc7b5fc-725tn_0e594c7f-af6f-432a-aec2-fd4399dbd462/kube-auth-proxy/0.log"
Sep 08 16:18:38.082113 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:38.082017    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ksf69_e8c13238-ca5b-4de2-a21f-4f187ca413b6/istio-proxy/0.log"
Sep 08 16:18:38.098314 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:38.098290    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ksf69_e8c13238-ca5b-4de2-a21f-4f187ca413b6/istio-proxy/1.log"
Sep 08 16:18:38.168297 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:38.168269    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-9g54m_26b382fb-6099-4697-91f1-c4d30851a597/payload-processing/0.log"
Sep 08 16:18:41.654800 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:41.654772    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5xsk_5d237297-4138-4c04-9495-cccbbff1e35e/kube-multus-additional-cni-plugins/0.log"
Sep 08 16:18:41.672196 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:41.672165    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5xsk_5d237297-4138-4c04-9495-cccbbff1e35e/egress-router-binary-copy/0.log"
Sep 08 16:18:41.689699 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:41.689634    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5xsk_5d237297-4138-4c04-9495-cccbbff1e35e/cni-plugins/0.log"
Sep 08 16:18:41.708647 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:41.708619    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5xsk_5d237297-4138-4c04-9495-cccbbff1e35e/bond-cni-plugin/0.log"
Sep 08 16:18:41.726516 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:41.726488    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5xsk_5d237297-4138-4c04-9495-cccbbff1e35e/routeoverride-cni/0.log"
Sep 08 16:18:41.743294 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:41.743271    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5xsk_5d237297-4138-4c04-9495-cccbbff1e35e/whereabouts-cni-bincopy/0.log"
Sep 08 16:18:41.760196 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:41.760168    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5xsk_5d237297-4138-4c04-9495-cccbbff1e35e/whereabouts-cni/0.log"
Sep 08 16:18:42.023568 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.023484    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s229p_1aa25f67-3646-4a1c-8b1b-d4117e8e4898/kube-multus/0.log"
Sep 08 16:18:42.044598 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.044558    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9jph2_37acb33a-d9a0-484e-94ed-69d21bbb302e/network-metrics-daemon/0.log"
Sep 08 16:18:42.057049 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.057015    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9jph2_37acb33a-d9a0-484e-94ed-69d21bbb302e/kube-rbac-proxy/0.log"
Sep 08 16:18:42.917133 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.917100    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/ovn-controller/0.log"
Sep 08 16:18:42.930504 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.930482    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/ovn-acl-logging/0.log"
Sep 08 16:18:42.948491 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.948469    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/kube-rbac-proxy-node/0.log"
Sep 08 16:18:42.965057 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.965005    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/kube-rbac-proxy-ovn-metrics/0.log"
Sep 08 16:18:42.981581 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.981561    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/northd/0.log"
Sep 08 16:18:42.997456 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:42.997433    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/nbdb/0.log"
Sep 08 16:18:43.014864 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:43.014844    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/sbdb/0.log"
Sep 08 16:18:43.096812 ip-10-0-136-224 kubenswrapper[2552]: I0908 16:18:43.096779    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ltprd_e9788893-de8d-4fef-bcde-772bd40a940e/ovnkube-controller/0.log"