Jul 24 16:43:11.663574 ip-10-0-135-204 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 16:43:11.954282 ip-10-0-135-204 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 16:43:11.954282 ip-10-0-135-204 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 16:43:11.954282 ip-10-0-135-204 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 16:43:11.954282 ip-10-0-135-204 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 16:43:11.954282 ip-10-0-135-204 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 16:43:11.955594 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.955508    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 16:43:11.959343 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959329    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 16:43:11.959343 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959344    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959348    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959351    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959355    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959358    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959361    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959364    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959371    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959374    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959379    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959382    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959386    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959389    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959392    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959395    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959398    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959401    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959404    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959407    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 16:43:11.959405 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959410    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959413    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959417    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959420    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959423    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959426    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959429    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959431    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959434    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959436    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959439    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959442    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959444    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959447    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959449    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959453    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959456    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959458    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959461    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 16:43:11.959860 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959465    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959468    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959471    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959474    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959477    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959480    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959482    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959485    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959487    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959490    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959492    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959495    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959498    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959501    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959503    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959506    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959509    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959511    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959514    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959517    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 16:43:11.960348 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959519    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959522    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959524    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959527    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959530    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959532    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959536    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959539    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959541    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959544    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959546    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959549    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959551    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959554    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959556    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959559    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959562    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959564    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959568    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 16:43:11.960876 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959572    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959575    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959577    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959580    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959582    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959585    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959588    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959591    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959594    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959596    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959599    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959601    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959604    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959606    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959609    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959612    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959614    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959617    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959619    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959622    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 16:43:11.961353 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.959625    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960902    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960911    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960914    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960918    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960921    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960924    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960927    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960930    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960933    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960936    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960939    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960942    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960944    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960947    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960950    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960952    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960955    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960958    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960961    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 16:43:11.961861 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960964    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960968    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960970    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960973    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960976    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960978    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960981    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960983    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960986    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960989    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960992    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960994    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.960997    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961000    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961004    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961006    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961009    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961013    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961017    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 16:43:11.962386 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961021    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961024    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961027    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961030    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961033    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961036    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961038    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961041    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961044    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961047    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961051    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961054    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961057    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961061    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961064    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961067    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961070    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961073    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961076    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 16:43:11.962884 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961079    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961082    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961085    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961088    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961091    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961094    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961096    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961099    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961101    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961104    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961107    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961109    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961114    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961117    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961120    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961122    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961125    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961128    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961131    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961134    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 16:43:11.963366 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961136    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961139    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961141    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961144    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961147    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961149    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961152    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961155    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961157    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961160    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961162    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961165    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961167    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961170    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961173    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961176    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961178    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961181    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961183    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961186    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 16:43:11.963858 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961189    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961191    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961597    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961603    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961606    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961609    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961611    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961614    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961617    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961620    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961623    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961626    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961628    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961631    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961634    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961637    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961639    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961643    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961646    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961649    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 16:43:11.964368 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961651    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961654    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961657    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961659    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961664    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961668    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961671    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961674    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961677    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961681    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961684    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961687    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961690    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961693    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961696    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961698    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961701    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961703    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961706    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 16:43:11.964854 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961709    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961713    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961717    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961720    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961723    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961726    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961728    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961731    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961733    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961736    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961740    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961742    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961745    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961747    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961750    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961752    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961755    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961757    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961760    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961762    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 16:43:11.965352 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961765    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961768    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961770    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961772    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961775    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961778    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961780    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961783    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961785    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961788    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961790    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961793    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961795    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961798    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961801    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961804    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961807    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961809    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961813    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961815    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 16:43:11.965835 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961818    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961821    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961824    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961826    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961829    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961831    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961834    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961836    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961839    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961842    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961844    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961847    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961850    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961852    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961855    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961857    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961860    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961863    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961866    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961868    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 16:43:11.966367 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961871    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.961874    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961940    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961947    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961953    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961958    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961970    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961974    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961978    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961982    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961986    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961989    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961993    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.961996    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962000    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962003    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962006    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962009    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962012    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962015    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962020    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962023    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962026    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962029    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 16:43:11.966856 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962033    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962036    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962040    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962043    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962046    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962050    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962053    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962056    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962059    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962062    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962066    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962069    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962072    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962079    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962082    2569 flags.go:64] FLAG: --enable-server="true"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962085    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962089    2569 flags.go:64] FLAG: --event-burst="100"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962093    2569 flags.go:64] FLAG: --event-qps="50"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962096    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962099    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962102    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962106    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962109    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962112    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962115    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 24 16:43:11.967444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962119    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962121    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962125    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962128    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962130    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962134    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962136    2569 flags.go:64] FLAG: --feature-gates=""
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962140    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962143    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962146    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962150    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962153    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962156    2569 flags.go:64] FLAG: --help="false"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962159    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-135-204.ec2.internal"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962162    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962165    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962168    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962171    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962175    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962178    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962182    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962185    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962188    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 16:43:11.968084 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962191    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962194    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962197    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962201    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962204    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962207    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962214    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962218    2569 flags.go:64] FLAG: --lock-file=""
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962221    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962224    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962227    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962232    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962235    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962238    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962241    2569 flags.go:64] FLAG: --logging-format="text"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962244    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962247    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962261    2569 flags.go:64] FLAG: --manifest-url=""
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962265    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962269    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962272    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962276    2569 flags.go:64] FLAG: --max-pods="110"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962280    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962283    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962286    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 16:43:11.968714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962289    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962292    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962295    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962298    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962306    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962310    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962313    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962316    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962319    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962324    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962327    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962331    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962334    2569 flags.go:64] FLAG: --port="10250"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962338    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962341    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5c5916f0d85510c"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962344    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962347    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962364    2569 flags.go:64] FLAG: --register-node="true"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962368    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962373    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962376    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962379    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962382    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962385    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962389    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 16:43:11.969654 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962392    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962395    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962399    2569 flags.go:64] FLAG: --runonce="false"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962402    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962406    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962409    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962412    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962415    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962418    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962421    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962424    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962427    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962431    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962434    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962437    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962440    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962443    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962446    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962451    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962455    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962460    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962464    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962467    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962470    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962473    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 16:43:11.970587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962476    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962479    2569 flags.go:64] FLAG: --v="2"
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962483    2569 flags.go:64] FLAG: --version="false"
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962487    2569 flags.go:64] FLAG: --vmodule=""
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962490    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.962493    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962609    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962614    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962618    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962621    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962624    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962627    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962629    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962632    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962635    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962637    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962640    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962643    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962646    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962650    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962652    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 16:43:11.971217 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962655    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962658    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962660    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962663    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962666    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962669    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962673    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962676    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962679    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962681    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962684    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962687    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962690    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962693    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962696    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962699    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962701    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962704    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962706    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 16:43:11.971730 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962709    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962711    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962714    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962717    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962719    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962722    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962725    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962727    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962730    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962732    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962735    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962740    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962743    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962745    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962748    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962751    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962753    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962756    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962759    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962763    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 16:43:11.972218 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962766    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962768    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962771    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962774    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962776    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962779    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962782    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962785    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962787    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962790    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962792    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962795    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962798    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962800    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962804    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962808    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962811    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962814    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962817    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962820    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 16:43:11.972738 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962823    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962826    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962829    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962833    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962836    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962838    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962841    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962844    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962846    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962850    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962853    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962858    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962861    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962864    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962867    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962870    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962873    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962875    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962878    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 16:43:11.973227 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962881    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962883    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962886    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962889    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962891    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.962894    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.963384    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.971082    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.971098    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.971920    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.971932    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.971993    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.971997    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972001    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972004    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 16:43:11.973702 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972007    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972010    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972013    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972016    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972019    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972022    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972024    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972027    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972030    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972033    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972035    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972038    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972041    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972043    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972046    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972049    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972051    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972054    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972057    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 16:43:11.974079 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972061    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972063    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972066    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972068    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972072    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972075    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972078    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972081    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972084    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972088    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972093    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972096    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972099    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972102    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972105    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972108    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972111    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972114    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972116    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 16:43:11.974604 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972119    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972122    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972124    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972127    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972129    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972132    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972135    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972138    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972141    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972143    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972146    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972148    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972151    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972154    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972156    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972159    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972161    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972165    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972167    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 16:43:11.975101 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972171    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972174    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972176    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972179    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972182    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972185    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972188    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972190    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972193    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972196    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972198    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972201    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972204    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972206    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972209    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972212    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972214    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972218    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972223    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972226    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 16:43:11.975577 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972229    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972231    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972234    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972237    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972239    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972242    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972245    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972248    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972263    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972266    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972269    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972272    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972275    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972278    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972280    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972283    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972286    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 16:43:11.976080 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972288    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.972294    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972412    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972418    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972421    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972424    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972427    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972429    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972432    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972435    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972438    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972441    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972443    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972446    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972449    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 16:43:11.976625 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972452    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972454    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972457    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972460    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972462    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972464    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972467    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972470    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972472    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972476    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972480    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972484    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972487    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972490    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972493    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972495    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972498    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972501    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972504    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 16:43:11.977091 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972507    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972510    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972513    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972515    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972518    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972520    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972523    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972526    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972528    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972531    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972533    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972536    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972538    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972542    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972544    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972547    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972550    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972552    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972555    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972557    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 16:43:11.977682 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972560    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972563    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972566    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972568    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972572    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972574    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972577    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972581    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972584    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972587    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972590    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972593    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972596    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972599    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972602    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972604    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972607    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972610    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972612    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972615    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 16:43:11.978505 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972617    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972620    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972622    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972625    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972627    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972630    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972633    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972636    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972639    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972641    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972644    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972646    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972649    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972652    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972655    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972657    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972664    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972667    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972670    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 16:43:11.979014 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972673    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972675    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972678    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972681    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972683    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972686    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972688    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:11.972691    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.972697    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.972827    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.974760    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.975978    2569 server.go:1019] "Starting client certificate rotation"
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.976103    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 16:43:11.979735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.976201    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 16:43:11.980082 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.978383    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 16:43:11.980082 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.979575    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 16:43:11.987307 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.987290    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 24 16:43:11.989233 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.989216    2569 log.go:25] "Validated CRI v1 image API"
Jul 24 16:43:11.990022 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.990007    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 16:43:11.995174 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.995152    2569 fs.go:135] Filesystem UUIDs: map[5474f7f0-7eca-47ef-a18a-dea4b3c66190:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a3fa710d-22e7-4dda-ab98-decfd9b50ec7:/dev/nvme0n1p4]
Jul 24 16:43:11.995276 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:11.995170    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 16:43:12.001757 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.001638    2569 manager.go:216] Machine: {Timestamp:2026-07-24 16:43:12.000124001 +0000 UTC m=+0.246009301 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099426 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2daf21b17800233a87cc0fc4f4bb09 SystemUUID:ec2daf21-b178-0023-3a87-cc0fc4f4bb09 BootID:eb7f5007-52c6-408d-a10e-eadd57aeccf4 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9f:45:4b:a7:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9f:45:4b:a7:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:2d:46:e2:79:4e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 16:43:12.001757 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.001745    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 16:43:12.001915 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.001831    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 16:43:12.002643 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.002609    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 16:43:12.002807 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.002647    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 16:43:12.003239 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.003229    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 16:43:12.003286 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.003241    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 16:43:12.003286 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.003259    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 16:43:12.003768 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.003757    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 16:43:12.004548 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.004537    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 16:43:12.007123 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007111    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 16:43:12.007280 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007270    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 16:43:12.007317 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007302    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 16:43:12.007317 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007312    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 16:43:12.007372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007323    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 16:43:12.007372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007331    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 24 16:43:12.007372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007341    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 16:43:12.007743 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007725    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 16:43:12.007797 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.007750    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 16:43:12.008871 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.008851    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 16:43:12.009488 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.009470    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 16:43:12.011240 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.011225    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 16:43:12.011312 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.011283    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 16:43:12.012618 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012606    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 16:43:12.012618 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012626    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012636    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012643    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012655    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012663    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012674    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012682    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012691    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012698    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 16:43:12.012714 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012710    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 16:43:12.012936 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.012720    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 16:43:12.013344 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.013334    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 16:43:12.013374 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.013346    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 16:43:12.017044 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.017028    2569 server.go:1268] "Started kubelet"
Jul 24 16:43:12.017126 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.017098    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 16:43:12.017637 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.017231    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 16:43:12.017637 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.017391    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 16:43:12.017843 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.017823    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 16:43:12.017874 ip-10-0-135-204 systemd[1]: Started Kubernetes Kubelet.
Jul 24 16:43:12.018135 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.018023    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 16:43:12.018135 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.018021    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 16:43:12.019837 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.019737    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 16:43:12.019837 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.019765    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 16:43:12.021640 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.020913    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-204.ec2.internal.18c5477444b8a725  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-204.ec2.internal,UID:ip-10-0-135-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-204.ec2.internal,},FirstTimestamp:2026-07-24 16:43:12.017000229 +0000 UTC m=+0.262885494,LastTimestamp:2026-07-24 16:43:12.017000229 +0000 UTC m=+0.262885494,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-204.ec2.internal,}"
Jul 24 16:43:12.024076 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.024055    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 16:43:12.025884 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.025870    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 16:43:12.025945 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.025890    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 16:43:12.026013 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.025996    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:12.026390 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026371    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 16:43:12.026476 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026421    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 16:43:12.026476 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026437    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 16:43:12.026713 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026696    2569 factory.go:55] Registering systemd factory
Jul 24 16:43:12.026713 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026712    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 24 16:43:12.026964 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026946    2569 factory.go:153] Registering CRI-O factory
Jul 24 16:43:12.027030 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026966    2569 factory.go:223] Registration of the crio container factory successfully
Jul 24 16:43:12.027030 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.026998    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 16:43:12.027030 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.027005    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 16:43:12.027030 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.027019    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 16:43:12.027211 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.027043    2569 factory.go:103] Registering Raw factory
Jul 24 16:43:12.027211 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.027057    2569 manager.go:1196] Started watching for new ooms in manager
Jul 24 16:43:12.027396 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.027371    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 16:43:12.027502 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.027484    2569 manager.go:349] Starting recovery of all containers
Jul 24 16:43:12.027502 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.027491    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 16:43:12.031562 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.031537    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2njjk"
Jul 24 16:43:12.038663 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.038647    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2njjk"
Jul 24 16:43:12.040548 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.040535    2569 manager.go:354] Recovery completed
Jul 24 16:43:12.045324 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.045310    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 16:43:12.047186 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.047172    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 16:43:12.047274 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.047199    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 16:43:12.047274 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.047211    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientPID"
Jul 24 16:43:12.047743 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.047722    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 16:43:12.047743 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.047736    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 16:43:12.047833 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.047751    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 16:43:12.047833 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.047763    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 16:43:12.048481 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.048468    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 16:43:12.048536 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.048492    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 16:43:12.048536 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.048502    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientPID"
Jul 24 16:43:12.050297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.050286    2569 policy_none.go:49] "None policy: Start"
Jul 24 16:43:12.050341 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.050301    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 16:43:12.050341 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.050313    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 16:43:12.052269 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.052246    2569 policy_none.go:47] "Start"
Jul 24 16:43:12.083130 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.082994    2569 manager.go:337] "Starting Device Plugin manager"
Jul 24 16:43:12.083213 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.083151    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 16:43:12.083213 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.083165    2569 server.go:87] "Starting device plugin registration server"
Jul 24 16:43:12.083421 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.083406    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 16:43:12.083498 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.083422    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 16:43:12.083553 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.083508    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 16:43:12.083599 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.083580    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 16:43:12.083599 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.083589    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 16:43:12.084122 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.084105    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 16:43:12.084197 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.084151    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:12.135865 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.135835    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 16:43:12.145361 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.137467    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 16:43:12.145361 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.137508    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 16:43:12.145361 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.137528    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 16:43:12.145361 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.137561    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 16:43:12.145361 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.139566    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 16:43:12.184326 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.184279    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 16:43:12.185277 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.185242    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 16:43:12.185372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.185286    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 16:43:12.185372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.185298    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientPID"
Jul 24 16:43:12.185372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.185322    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.193504 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.193487    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.193570 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.193511    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-204.ec2.internal\": node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:12.207505 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.207440    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:12.237985 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.237956    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal"]
Jul 24 16:43:12.238119 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.238023    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 16:43:12.239477 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.239459    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 16:43:12.239565 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.239497    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 16:43:12.239565 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.239513    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientPID"
Jul 24 16:43:12.241931 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.241917    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 16:43:12.242028 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242015    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.242075 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242042    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 16:43:12.242680 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242661    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 16:43:12.242790 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242687    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 16:43:12.242790 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242700    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientPID"
Jul 24 16:43:12.242790 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242712    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 16:43:12.242790 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242730    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 16:43:12.242790 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.242739    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientPID"
Jul 24 16:43:12.244993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.244977    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.245085 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.245002    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 16:43:12.245709 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.245694    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 16:43:12.245787 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.245720    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 16:43:12.245787 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.245733    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeHasSufficientPID"
Jul 24 16:43:12.277957 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.277927    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-204.ec2.internal\" not found" node="ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.282379 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.282362    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-204.ec2.internal\" not found" node="ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.308296 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.308275    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:12.329012 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.328991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9cfb6a4f985f47da5766891ed8f37454-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal\" (UID: \"9cfb6a4f985f47da5766891ed8f37454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.329080 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.329015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfb6a4f985f47da5766891ed8f37454-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal\" (UID: \"9cfb6a4f985f47da5766891ed8f37454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.329080 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.329033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb8794d5494e853f2db83757f0e6f8d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-204.ec2.internal\" (UID: \"aeb8794d5494e853f2db83757f0e6f8d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.409370 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.409332    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:12.429670 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.429644    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9cfb6a4f985f47da5766891ed8f37454-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal\" (UID: \"9cfb6a4f985f47da5766891ed8f37454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.429728 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.429673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfb6a4f985f47da5766891ed8f37454-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal\" (UID: \"9cfb6a4f985f47da5766891ed8f37454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.429728 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.429690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb8794d5494e853f2db83757f0e6f8d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-204.ec2.internal\" (UID: \"aeb8794d5494e853f2db83757f0e6f8d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.429728 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.429723    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cfb6a4f985f47da5766891ed8f37454-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal\" (UID: \"9cfb6a4f985f47da5766891ed8f37454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.429825 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.429731    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9cfb6a4f985f47da5766891ed8f37454-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal\" (UID: \"9cfb6a4f985f47da5766891ed8f37454\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.429825 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.429754    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aeb8794d5494e853f2db83757f0e6f8d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-204.ec2.internal\" (UID: \"aeb8794d5494e853f2db83757f0e6f8d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.510097 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:12.510030    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:12.580590 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.580564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:12.585489 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:12.585462    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:14.097138 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:14.097108    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:14.197661 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:14.197629    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:14.298180 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:14.298103    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:14.398797 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:14.398768    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:14.461193 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.461161    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 16:43:14.461651 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.461349    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 16:43:14.499598 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:14.499566    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:14.514435 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.514401    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 16:43:14.530261 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.530228    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 16:38:13 +0000 UTC" deadline="2028-01-31 01:26:47.165412946 +0000 UTC"
Jul 24 16:43:14.530261 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.530258    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13328h43m32.635159004s"
Jul 24 16:43:14.600642 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:14.600563    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:14.669277 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:14.669246    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfb6a4f985f47da5766891ed8f37454.slice/crio-fa8a5441613df437e3c42020af4f240b969258a5f62cb96f38b22bdb53581a66 WatchSource:0}: Error finding container fa8a5441613df437e3c42020af4f240b969258a5f62cb96f38b22bdb53581a66: Status 404 returned error can't find the container with id fa8a5441613df437e3c42020af4f240b969258a5f62cb96f38b22bdb53581a66
Jul 24 16:43:14.669662 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:14.669641    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb8794d5494e853f2db83757f0e6f8d.slice/crio-b5367c82060a98754e76c22e4e819fa9f9c2da7524fe8b5b56e5173254968ab1 WatchSource:0}: Error finding container b5367c82060a98754e76c22e4e819fa9f9c2da7524fe8b5b56e5173254968ab1: Status 404 returned error can't find the container with id b5367c82060a98754e76c22e4e819fa9f9c2da7524fe8b5b56e5173254968ab1
Jul 24 16:43:14.673007 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.672992    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:43:14.701271 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:14.701228    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-204.ec2.internal\" not found"
Jul 24 16:43:14.751575 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.751551    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 16:43:14.812153 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.812125    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:14.823872 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.823847    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 16:43:14.824605 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.824592    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal"
Jul 24 16:43:14.832583 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.832564    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 16:43:14.932472 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:14.932396    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 16:43:15.495464 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.495424    2569 apiserver.go:52] "Watching apiserver"
Jul 24 16:43:15.501648 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.501609    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 16:43:15.502686 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.502656    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-2mvp5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal","openshift-multus/multus-additional-cni-plugins-8hzgt","openshift-multus/network-metrics-daemon-5mppg","kube-system/global-pull-secret-syncer-jmx8q","kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b","openshift-multus/multus-zld5x","openshift-network-diagnostics/network-check-target-chhhw","openshift-network-operator/iptables-alerter-wsjn9","openshift-ovn-kubernetes/ovnkube-node-wprhx","kube-system/konnectivity-agent-sx8db","openshift-cluster-node-tuning-operator/tuned-ptrlq","openshift-dns/node-resolver-8774l"]
Jul 24 16:43:15.505144 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.505120    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.507688 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.507665    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 16:43:15.507688 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.507678    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 16:43:15.507851 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.507716    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 16:43:15.507851 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.507668    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jshsg\""
Jul 24 16:43:15.511173 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.511133    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.512795 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.512779    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 16:43:15.513251 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.513229    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 16:43:15.513373 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.513303    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 16:43:15.513441 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.513413    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7sk2c\""
Jul 24 16:43:15.513643 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.513625    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.515759 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.515740    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 16:43:15.516500 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.515763    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 16:43:15.516500 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.515766    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 16:43:15.516500 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.515802    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 16:43:15.516500 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.515985    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 16:43:15.516500 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.516029    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:15.516771 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.516167    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7nlsp\""
Jul 24 16:43:15.516771 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.516531    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:15.518771 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.518736    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.518886 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.518842    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:15.520739 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.520713    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.522864 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.522812    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 16:43:15.523674 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.523119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:15.523674 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.523182    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:15.523674 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.523386    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v4j7c\""
Jul 24 16:43:15.523674 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.523506    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 16:43:15.525443 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.525424    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.525941 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.525919    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 16:43:15.526154 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-socket-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.526232 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfvxl\" (UniqueName: \"kubernetes.io/projected/74780675-2b53-4d56-9b16-2d1f7e718afa-kube-api-access-mfvxl\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.526232 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-modprobe-d\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526318 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25878a83-561c-45f2-9dc2-96d20c354f1e-tmp\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526318 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-system-cni-dir\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.526475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-registration-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.526475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-sys-fs\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.526475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-lib-modules\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526436    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-var-lib-kubelet\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-tuned\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526479    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysctl-d\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526499    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-run\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-sys\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526604    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526636    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-systemd\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cnibin\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526689    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.526730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpvsx\" (UniqueName: \"kubernetes.io/projected/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-kube-api-access-hpvsx\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526741    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-device-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-etc-selinux\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-kubernetes\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-host\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526914    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysconfig\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysctl-conf\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.526976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf6sx\" (UniqueName: \"kubernetes.io/projected/25878a83-561c-45f2-9dc2-96d20c354f1e-kube-api-access-rf6sx\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.527011    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-os-release\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.527078 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.527036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.528230 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.528208    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dp5dv\""
Jul 24 16:43:15.528544 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.528524    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 16:43:15.529435 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.529140    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 16:43:15.529641 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.529617    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 16:43:15.531950 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.531927    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.533969 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.533950    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 16:43:15.534158 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.534134    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w9gzx\""
Jul 24 16:43:15.534439 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.534410    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.536865 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.536684    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 16:43:15.536865 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.536849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wcpjh\""
Jul 24 16:43:15.537010 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.536852    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 16:43:15.537442 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.537400    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 16:43:15.537897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.537879    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.540127 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.540103    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 16:43:15.540240 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.540165    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 16:43:15.540240 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.540213    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cxqsg\""
Jul 24 16:43:15.540358 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.540103    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 16:43:15.540415 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.540372    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 16:43:15.540566 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.540550    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:15.541140 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.541118    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 16:43:15.541239 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.541215    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 16:43:15.542870 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.542563    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 16:43:15.542870 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.542665    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 16:43:15.542870 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.542570    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2s7z6\""
Jul 24 16:43:15.544993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.544971    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4v59h"
Jul 24 16:43:15.551192 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.550835    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4v59h"
Jul 24 16:43:15.614217 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.614193    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 16:43:15.627208 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cnibin\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.627374 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.627374 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-kubelet\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.627374 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.627374 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cnibin\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.627374 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-ovnkube-script-lib\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.627374 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-host\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.627686 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-etc-selinux\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.627686 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.627686 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-etc-kubernetes\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.627686 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627463    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-host\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.627686 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81d06610-c1f7-4523-9af8-7691ca687a7e-ovn-node-metrics-cert\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.627686 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627508    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6j27\" (UniqueName: \"kubernetes.io/projected/81d06610-c1f7-4523-9af8-7691ca687a7e-kube-api-access-l6j27\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.627866 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627545    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-etc-selinux\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.627866 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.627533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf6sx\" (UniqueName: \"kubernetes.io/projected/25878a83-561c-45f2-9dc2-96d20c354f1e-kube-api-access-rf6sx\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.628268 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628238    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.628386 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-host\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.628475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628446    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.628583 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628562    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-os-release\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.628655 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628600    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03ab4509-d158-4689-93c5-10571b0062f7-cni-binary-copy\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.628655 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628636    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-socket-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.628757 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-system-cni-dir\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.628757 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628697    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-system-cni-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.628757 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628726    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-hostroot\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.628897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628775    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-kubelet\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.628897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628775    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.628897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628854    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.628897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-system-cni-dir\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.629101 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-netns\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.629101 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-conf-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.629101 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.629244 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.629499 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629362    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal" event={"ID":"9cfb6a4f985f47da5766891ed8f37454","Type":"ContainerStarted","Data":"fa8a5441613df437e3c42020af4f240b969258a5f62cb96f38b22bdb53581a66"}
Jul 24 16:43:15.629633 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.628878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-socket-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.629753 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629695    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d5c331c-2d1f-481b-a958-080182561712-kubelet-config\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.629753 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629732    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.629851 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1de7e8c-5014-4551-8158-d1c2101f616a-hosts-file\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.629851 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629793    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-sys\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.629851 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629823    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-cni-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.629988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629851    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-env-overrides\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.629988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629872    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-sys\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.629988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7gd\" (UniqueName: \"kubernetes.io/projected/03ab4509-d158-4689-93c5-10571b0062f7-kube-api-access-fz7gd\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.629988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-systemd-units\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.629988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.629973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-slash\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.630194 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-run-netns\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.630194 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b3b84e6-8b95-41de-89fe-b68b7158ebec-konnectivity-ca\") pod \"konnectivity-agent-sx8db\" (UID: \"9b3b84e6-8b95-41de-89fe-b68b7158ebec\") " pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:15.630194 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630073    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d5c331c-2d1f-481b-a958-080182561712-dbus\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.630194 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-systemd\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.630194 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630135    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpvsx\" (UniqueName: \"kubernetes.io/projected/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-kube-api-access-hpvsx\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.630194 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-cnibin\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-multus-certs\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630223    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/56bf2e0a-2c83-4387-8641-dab90c57c69f-iptables-alerter-script\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630250    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56bf2e0a-2c83-4387-8641-dab90c57c69f-host-slash\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630277    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1de7e8c-5014-4551-8158-d1c2101f616a-tmp-dir\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-device-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-socket-dir-parent\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-systemd\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630440    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-systemd\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630447    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-var-lib-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.630487 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-ovnkube-config\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-kubernetes\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630532    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03ab4509-d158-4689-93c5-10571b0062f7-multus-daemon-config\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj956\" (UniqueName: \"kubernetes.io/projected/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-kube-api-access-qj956\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-device-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysconfig\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysctl-conf\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-os-release\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p9lf\" (UniqueName: \"kubernetes.io/projected/56bf2e0a-2c83-4387-8641-dab90c57c69f-kube-api-access-5p9lf\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630692    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-log-socket\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b3b84e6-8b95-41de-89fe-b68b7158ebec-agent-certs\") pod \"konnectivity-agent-sx8db\" (UID: \"9b3b84e6-8b95-41de-89fe-b68b7158ebec\") " pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-kubernetes\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630773    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfvxl\" (UniqueName: \"kubernetes.io/projected/74780675-2b53-4d56-9b16-2d1f7e718afa-kube-api-access-mfvxl\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-os-release\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-modprobe-d\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.630975 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630908    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25878a83-561c-45f2-9dc2-96d20c354f1e-tmp\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysctl-conf\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-cni-netd\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.630981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-registration-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-sys-fs\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-lib-modules\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-registration-dir\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631144    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74780675-2b53-4d56-9b16-2d1f7e718afa-sys-fs\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysconfig\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631196    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-modprobe-d\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-var-lib-kubelet\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631357    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-tuned\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-cni-bin\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631416    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-cni-multus\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631396    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-lib-modules\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.631744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631570    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-var-lib-kubelet\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.631489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-node-log\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632049    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysctl-d\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-run\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-k8s-cni-cncf-io\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632164    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-etc-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632192    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-cni-bin\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632248    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbgkh\" (UniqueName: \"kubernetes.io/projected/12520acc-f016-4ab9-bc6e-3efd878c185b-kube-api-access-rbgkh\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4lfd\" (UniqueName: \"kubernetes.io/projected/c1de7e8c-5014-4551-8158-d1c2101f616a-kube-api-access-h4lfd\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632323    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-ovn\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.632535 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632363    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-serviceca\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.633043 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-sysctl-d\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.633043 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632567    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25878a83-561c-45f2-9dc2-96d20c354f1e-run\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.633043 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632602    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.633043 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.632624    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal" event={"ID":"aeb8794d5494e853f2db83757f0e6f8d","Type":"ContainerStarted","Data":"b5367c82060a98754e76c22e4e819fa9f9c2da7524fe8b5b56e5173254968ab1"}
Jul 24 16:43:15.635064 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.635042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25878a83-561c-45f2-9dc2-96d20c354f1e-tmp\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.635176 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.635098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25878a83-561c-45f2-9dc2-96d20c354f1e-etc-tuned\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.635839 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.635816    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf6sx\" (UniqueName: \"kubernetes.io/projected/25878a83-561c-45f2-9dc2-96d20c354f1e-kube-api-access-rf6sx\") pod \"tuned-ptrlq\" (UID: \"25878a83-561c-45f2-9dc2-96d20c354f1e\") " pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.641168 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.641145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpvsx\" (UniqueName: \"kubernetes.io/projected/44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43-kube-api-access-hpvsx\") pod \"multus-additional-cni-plugins-8hzgt\" (UID: \"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43\") " pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.641576 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.641555    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfvxl\" (UniqueName: \"kubernetes.io/projected/74780675-2b53-4d56-9b16-2d1f7e718afa-kube-api-access-mfvxl\") pod \"aws-ebs-csi-driver-node-tp98b\" (UID: \"74780675-2b53-4d56-9b16-2d1f7e718afa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.732817 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732781    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-cnibin\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.732817 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-multus-certs\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/56bf2e0a-2c83-4387-8641-dab90c57c69f-iptables-alerter-script\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732863    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56bf2e0a-2c83-4387-8641-dab90c57c69f-host-slash\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1de7e8c-5014-4551-8158-d1c2101f616a-tmp-dir\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-socket-dir-parent\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732927    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-systemd\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732930    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-cnibin\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56bf2e0a-2c83-4387-8641-dab90c57c69f-host-slash\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732930    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-multus-certs\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-var-lib-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.732986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-var-lib-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-systemd\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733006    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-ovnkube-config\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733014    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-socket-dir-parent\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03ab4509-d158-4689-93c5-10571b0062f7-multus-daemon-config\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733053 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj956\" (UniqueName: \"kubernetes.io/projected/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-kube-api-access-qj956\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p9lf\" (UniqueName: \"kubernetes.io/projected/56bf2e0a-2c83-4387-8641-dab90c57c69f-kube-api-access-5p9lf\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733137    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-log-socket\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b3b84e6-8b95-41de-89fe-b68b7158ebec-agent-certs\") pod \"konnectivity-agent-sx8db\" (UID: \"9b3b84e6-8b95-41de-89fe-b68b7158ebec\") " pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733193    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-cni-netd\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733211    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-cni-bin\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-cni-multus\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-node-log\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-k8s-cni-cncf-io\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-etc-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-cni-bin\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733300    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733302    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1de7e8c-5014-4551-8158-d1c2101f616a-tmp-dir\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbgkh\" (UniqueName: \"kubernetes.io/projected/12520acc-f016-4ab9-bc6e-3efd878c185b-kube-api-access-rbgkh\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lfd\" (UniqueName: \"kubernetes.io/projected/c1de7e8c-5014-4551-8158-d1c2101f616a-kube-api-access-h4lfd\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-ovn\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.733789 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-serviceca\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733392    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-cni-bin\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733400    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-kubelet\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733427    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-kubelet\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733434    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733460    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-var-lib-cni-multus\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733499    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/56bf2e0a-2c83-4387-8641-dab90c57c69f-iptables-alerter-script\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733605    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03ab4509-d158-4689-93c5-10571b0062f7-multus-daemon-config\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.733703    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733766    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-cni-netd\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.733794    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:43:16.233755681 +0000 UTC m=+2.993564673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733822    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-run-ovn\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733836    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-log-socket\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-node-log\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733928    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-ovnkube-script-lib\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733940    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-k8s-cni-cncf-io\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.734635 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-ovnkube-script-lib\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733961    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-etc-openvswitch\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-host\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.733991    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-cni-bin\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-etc-kubernetes\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734030    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81d06610-c1f7-4523-9af8-7691ca687a7e-ovn-node-metrics-cert\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734062    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-etc-kubernetes\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6j27\" (UniqueName: \"kubernetes.io/projected/81d06610-c1f7-4523-9af8-7691ca687a7e-kube-api-access-l6j27\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734033    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-host\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-os-release\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03ab4509-d158-4689-93c5-10571b0062f7-cni-binary-copy\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734142    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-system-cni-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734166    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-hostroot\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-os-release\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734187    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-kubelet\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734208    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-ovnkube-config\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-system-cni-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734245    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-serviceca\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.735621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734264    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-netns\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734293    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-conf-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734305    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-host-run-netns\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734268    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-hostroot\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-conf-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734370    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-run-ovn-kubernetes\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734236    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-kubelet\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734379    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734420    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734462    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d5c331c-2d1f-481b-a958-080182561712-kubelet-config\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734516    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4d5c331c-2d1f-481b-a958-080182561712-kubelet-config\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.734568    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734573    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1de7e8c-5014-4551-8158-d1c2101f616a-hosts-file\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c1de7e8c-5014-4551-8158-d1c2101f616a-hosts-file\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734607    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-cni-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.736479 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-env-overrides\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.734637    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret podName:4d5c331c-2d1f-481b-a958-080182561712 nodeName:}" failed. No retries permitted until 2026-07-24 16:43:16.234620962 +0000 UTC m=+2.994429941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret") pod "global-pull-secret-syncer-jmx8q" (UID: "4d5c331c-2d1f-481b-a958-080182561712") : object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734666    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7gd\" (UniqueName: \"kubernetes.io/projected/03ab4509-d158-4689-93c5-10571b0062f7-kube-api-access-fz7gd\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-systemd-units\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734715    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-slash\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-run-netns\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734763    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b3b84e6-8b95-41de-89fe-b68b7158ebec-konnectivity-ca\") pod \"konnectivity-agent-sx8db\" (UID: \"9b3b84e6-8b95-41de-89fe-b68b7158ebec\") " pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734798    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03ab4509-d158-4689-93c5-10571b0062f7-multus-cni-dir\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d5c331c-2d1f-481b-a958-080182561712-dbus\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734850    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-slash\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-host-run-netns\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734886    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/81d06610-c1f7-4523-9af8-7691ca687a7e-systemd-units\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734905    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4d5c331c-2d1f-481b-a958-080182561712-dbus\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.734938    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/81d06610-c1f7-4523-9af8-7691ca687a7e-env-overrides\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.735132    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03ab4509-d158-4689-93c5-10571b0062f7-cni-binary-copy\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.735348    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b3b84e6-8b95-41de-89fe-b68b7158ebec-konnectivity-ca\") pod \"konnectivity-agent-sx8db\" (UID: \"9b3b84e6-8b95-41de-89fe-b68b7158ebec\") " pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.736175    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b3b84e6-8b95-41de-89fe-b68b7158ebec-agent-certs\") pod \"konnectivity-agent-sx8db\" (UID: \"9b3b84e6-8b95-41de-89fe-b68b7158ebec\") " pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:15.737020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.736530    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/81d06610-c1f7-4523-9af8-7691ca687a7e-ovn-node-metrics-cert\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.740933 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.740852    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 16:43:15.740933 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.740874    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 16:43:15.740933 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.740887    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q24vn for pod openshift-network-diagnostics/network-check-target-chhhw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:15.741169 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:15.740941    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn podName:9d026564-c995-470c-8186-7afe435c2b9f nodeName:}" failed. No retries permitted until 2026-07-24 16:43:16.240924257 +0000 UTC m=+3.000733255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q24vn" (UniqueName: "kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn") pod "network-check-target-chhhw" (UID: "9d026564-c995-470c-8186-7afe435c2b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:15.742540 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.742514    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p9lf\" (UniqueName: \"kubernetes.io/projected/56bf2e0a-2c83-4387-8641-dab90c57c69f-kube-api-access-5p9lf\") pod \"iptables-alerter-wsjn9\" (UID: \"56bf2e0a-2c83-4387-8641-dab90c57c69f\") " pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.742908 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.742881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbgkh\" (UniqueName: \"kubernetes.io/projected/12520acc-f016-4ab9-bc6e-3efd878c185b-kube-api-access-rbgkh\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:15.743261 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.743241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj956\" (UniqueName: \"kubernetes.io/projected/e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec-kube-api-access-qj956\") pod \"node-ca-2mvp5\" (UID: \"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec\") " pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.743261 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.743257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lfd\" (UniqueName: \"kubernetes.io/projected/c1de7e8c-5014-4551-8158-d1c2101f616a-kube-api-access-h4lfd\") pod \"node-resolver-8774l\" (UID: \"c1de7e8c-5014-4551-8158-d1c2101f616a\") " pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.743639 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.743620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7gd\" (UniqueName: \"kubernetes.io/projected/03ab4509-d158-4689-93c5-10571b0062f7-kube-api-access-fz7gd\") pod \"multus-zld5x\" (UID: \"03ab4509-d158-4689-93c5-10571b0062f7\") " pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.743757 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.743731    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6j27\" (UniqueName: \"kubernetes.io/projected/81d06610-c1f7-4523-9af8-7691ca687a7e-kube-api-access-l6j27\") pod \"ovnkube-node-wprhx\" (UID: \"81d06610-c1f7-4523-9af8-7691ca687a7e\") " pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.799656 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.799576    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 16:43:15.820677 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.820642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b"
Jul 24 16:43:15.829345 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.829297    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ptrlq"
Jul 24 16:43:15.842082 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.842063    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8hzgt"
Jul 24 16:43:15.847806 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.847786    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8774l"
Jul 24 16:43:15.855412 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.855390    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2mvp5"
Jul 24 16:43:15.863043 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.863025    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zld5x"
Jul 24 16:43:15.872705 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.872677    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wsjn9"
Jul 24 16:43:15.880460 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.880341    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:15.884405 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:15.883944    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:16.237496 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.237417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:16.237496 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.237461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:16.237704 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.237552    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:16.237704 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.237572    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:16.237704 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.237608    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:43:17.237593186 +0000 UTC m=+3.997402164 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:16.237704 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.237631    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret podName:4d5c331c-2d1f-481b-a958-080182561712 nodeName:}" failed. No retries permitted until 2026-07-24 16:43:17.237614571 +0000 UTC m=+3.997423564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret") pod "global-pull-secret-syncer-jmx8q" (UID: "4d5c331c-2d1f-481b-a958-080182561712") : object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:16.338173 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.338133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:16.338384 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.338300    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 16:43:16.338384 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.338319    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 16:43:16.338384 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.338343    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q24vn for pod openshift-network-diagnostics/network-check-target-chhhw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:16.338563 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.338408    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn podName:9d026564-c995-470c-8186-7afe435c2b9f nodeName:}" failed. No retries permitted until 2026-07-24 16:43:17.338392987 +0000 UTC m=+4.098201967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q24vn" (UniqueName: "kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn") pod "network-check-target-chhhw" (UID: "9d026564-c995-470c-8186-7afe435c2b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:16.553735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.553657    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 16:38:15 +0000 UTC" deadline="2028-04-23 02:23:01.76257297 +0000 UTC"
Jul 24 16:43:16.553735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.553684    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15321h39m45.208892412s"
Jul 24 16:43:16.593309 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.593270    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d06610_c1f7_4523_9af8_7691ca687a7e.slice/crio-9ae06b90dc4ddd8d8e2c5ee64dec479f6ca30a6dadfb1cabd9533bc0a2904eab WatchSource:0}: Error finding container 9ae06b90dc4ddd8d8e2c5ee64dec479f6ca30a6dadfb1cabd9533bc0a2904eab: Status 404 returned error can't find the container with id 9ae06b90dc4ddd8d8e2c5ee64dec479f6ca30a6dadfb1cabd9533bc0a2904eab
Jul 24 16:43:16.595263 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.595203    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ab4509_d158_4689_93c5_10571b0062f7.slice/crio-fe9a8266e6136d83f4421c396520e170ffdb54f96e82a8a865fa2507acfa58e1 WatchSource:0}: Error finding container fe9a8266e6136d83f4421c396520e170ffdb54f96e82a8a865fa2507acfa58e1: Status 404 returned error can't find the container with id fe9a8266e6136d83f4421c396520e170ffdb54f96e82a8a865fa2507acfa58e1
Jul 24 16:43:16.601074 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.601051    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56bf2e0a_2c83_4387_8641_dab90c57c69f.slice/crio-fa53783e312b7afd5947af48a944541ee2158b71fcc8741d9a4e885b16219d05 WatchSource:0}: Error finding container fa53783e312b7afd5947af48a944541ee2158b71fcc8741d9a4e885b16219d05: Status 404 returned error can't find the container with id fa53783e312b7afd5947af48a944541ee2158b71fcc8741d9a4e885b16219d05
Jul 24 16:43:16.602683 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.602387    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25878a83_561c_45f2_9dc2_96d20c354f1e.slice/crio-6e67d9178fba37159677a7c8f47b80c4d84a7344a9170e8f46b84860cdc6741e WatchSource:0}: Error finding container 6e67d9178fba37159677a7c8f47b80c4d84a7344a9170e8f46b84860cdc6741e: Status 404 returned error can't find the container with id 6e67d9178fba37159677a7c8f47b80c4d84a7344a9170e8f46b84860cdc6741e
Jul 24 16:43:16.603171 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.603057    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c14d78_a18c_4f2f_8f15_dc8bcbfbcd43.slice/crio-4d8996a76f0082fa4ea5ad6398c0c6451e7c4fa05a07df13d120a97e415b6808 WatchSource:0}: Error finding container 4d8996a76f0082fa4ea5ad6398c0c6451e7c4fa05a07df13d120a97e415b6808: Status 404 returned error can't find the container with id 4d8996a76f0082fa4ea5ad6398c0c6451e7c4fa05a07df13d120a97e415b6808
Jul 24 16:43:16.604255 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.604139    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74780675_2b53_4d56_9b16_2d1f7e718afa.slice/crio-db7c7042a13ae0f359bf17d2f49c31d85b48c4789bd8629277558ab8b0b63c22 WatchSource:0}: Error finding container db7c7042a13ae0f359bf17d2f49c31d85b48c4789bd8629277558ab8b0b63c22: Status 404 returned error can't find the container with id db7c7042a13ae0f359bf17d2f49c31d85b48c4789bd8629277558ab8b0b63c22
Jul 24 16:43:16.605356 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.605261    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3b84e6_8b95_41de_89fe_b68b7158ebec.slice/crio-bf7b6bf4fd2e81f3cb859b356ec440a3332ac878b92873df82d65c0a4f6679cc WatchSource:0}: Error finding container bf7b6bf4fd2e81f3cb859b356ec440a3332ac878b92873df82d65c0a4f6679cc: Status 404 returned error can't find the container with id bf7b6bf4fd2e81f3cb859b356ec440a3332ac878b92873df82d65c0a4f6679cc
Jul 24 16:43:16.606545 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:16.606456    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1de7e8c_5014_4551_8158_d1c2101f616a.slice/crio-b9069ac0933c3e889a79b7484ff824631a9d910f554cc281ff1dcf8a2f6d3f03 WatchSource:0}: Error finding container b9069ac0933c3e889a79b7484ff824631a9d910f554cc281ff1dcf8a2f6d3f03: Status 404 returned error can't find the container with id b9069ac0933c3e889a79b7484ff824631a9d910f554cc281ff1dcf8a2f6d3f03
Jul 24 16:43:16.623899 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.623875    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:16.623990 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:16.623967    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:16.635043 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.635016    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zld5x" event={"ID":"03ab4509-d158-4689-93c5-10571b0062f7","Type":"ContainerStarted","Data":"fe9a8266e6136d83f4421c396520e170ffdb54f96e82a8a865fa2507acfa58e1"}
Jul 24 16:43:16.636581 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.636555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sx8db" event={"ID":"9b3b84e6-8b95-41de-89fe-b68b7158ebec","Type":"ContainerStarted","Data":"bf7b6bf4fd2e81f3cb859b356ec440a3332ac878b92873df82d65c0a4f6679cc"}
Jul 24 16:43:16.638523 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.638496    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b" event={"ID":"74780675-2b53-4d56-9b16-2d1f7e718afa","Type":"ContainerStarted","Data":"db7c7042a13ae0f359bf17d2f49c31d85b48c4789bd8629277558ab8b0b63c22"}
Jul 24 16:43:16.642568 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.642546    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerStarted","Data":"4d8996a76f0082fa4ea5ad6398c0c6451e7c4fa05a07df13d120a97e415b6808"}
Jul 24 16:43:16.645842 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.645820    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ptrlq" event={"ID":"25878a83-561c-45f2-9dc2-96d20c354f1e","Type":"ContainerStarted","Data":"6e67d9178fba37159677a7c8f47b80c4d84a7344a9170e8f46b84860cdc6741e"}
Jul 24 16:43:16.647134 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.647115    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wsjn9" event={"ID":"56bf2e0a-2c83-4387-8641-dab90c57c69f","Type":"ContainerStarted","Data":"fa53783e312b7afd5947af48a944541ee2158b71fcc8741d9a4e885b16219d05"}
Jul 24 16:43:16.648276 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.648251    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"9ae06b90dc4ddd8d8e2c5ee64dec479f6ca30a6dadfb1cabd9533bc0a2904eab"}
Jul 24 16:43:16.649371 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.649349    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2mvp5" event={"ID":"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec","Type":"ContainerStarted","Data":"be42973cc5db5538ef159e57bc5fc0fc78afe686e3e51fe0ddc3371356da918e"}
Jul 24 16:43:16.650237 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:16.650217    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8774l" event={"ID":"c1de7e8c-5014-4551-8158-d1c2101f616a","Type":"ContainerStarted","Data":"b9069ac0933c3e889a79b7484ff824631a9d910f554cc281ff1dcf8a2f6d3f03"}
Jul 24 16:43:17.244810 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.244387    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:17.244969 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.244816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:17.244969 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.244556    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:17.245106 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.245024    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret podName:4d5c331c-2d1f-481b-a958-080182561712 nodeName:}" failed. No retries permitted until 2026-07-24 16:43:19.244992597 +0000 UTC m=+6.004801577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret") pod "global-pull-secret-syncer-jmx8q" (UID: "4d5c331c-2d1f-481b-a958-080182561712") : object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:17.245174 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.244947    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:17.245226 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.245198    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:43:19.245176369 +0000 UTC m=+6.004985356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:17.346105 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.345446    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:17.346105 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.345651    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 16:43:17.346105 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.345669    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 16:43:17.346105 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.345683    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q24vn for pod openshift-network-diagnostics/network-check-target-chhhw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:17.346105 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.345740    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn podName:9d026564-c995-470c-8186-7afe435c2b9f nodeName:}" failed. No retries permitted until 2026-07-24 16:43:19.345721328 +0000 UTC m=+6.105530311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q24vn" (UniqueName: "kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn") pod "network-check-target-chhhw" (UID: "9d026564-c995-470c-8186-7afe435c2b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:17.554098 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.553987    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 16:38:15 +0000 UTC" deadline="2028-03-02 02:39:59.115557043 +0000 UTC"
Jul 24 16:43:17.554098 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.554047    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14073h56m41.56151433s"
Jul 24 16:43:17.626423 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.626358    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:17.626590 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.626516    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:17.626993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.626972    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:17.627098 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:17.627069    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:17.665803 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.665764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal" event={"ID":"aeb8794d5494e853f2db83757f0e6f8d","Type":"ContainerStarted","Data":"687c85ae1b7d814a0c80daa3e5e53203a81ae285efc7043cfc358556c82bb117"}
Jul 24 16:43:17.680511 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.679631    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-204.ec2.internal" podStartSLOduration=3.679586269 podStartE2EDuration="3.679586269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:17.679249179 +0000 UTC m=+4.439058174" watchObservedRunningTime="2026-07-24 16:43:17.679586269 +0000 UTC m=+4.439395268"
Jul 24 16:43:17.682671 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.682047    2569 generic.go:356] "Generic (PLEG): container finished" podID="9cfb6a4f985f47da5766891ed8f37454" containerID="489b9e1a412790e41349fed1c7a41777c509716725a08798d69b1e652799dedf" exitCode=0
Jul 24 16:43:17.682671 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:17.682095    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal" event={"ID":"9cfb6a4f985f47da5766891ed8f37454","Type":"ContainerDied","Data":"489b9e1a412790e41349fed1c7a41777c509716725a08798d69b1e652799dedf"}
Jul 24 16:43:18.624477 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:18.624388    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:18.625019 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:18.624525    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:18.706842 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:18.706403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal" event={"ID":"9cfb6a4f985f47da5766891ed8f37454","Type":"ContainerStarted","Data":"6fa5e7db36fad09cdad1f7a0237795ad752feeb8664fb98d4fc5ba148c967279"}
Jul 24 16:43:19.261345 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:19.261292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:19.261521 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:19.261376    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:19.261598 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.261532    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:19.261598 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.261593    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:43:23.261573597 +0000 UTC m=+10.021382590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:19.262023 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.262000    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:19.262130 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.262063    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret podName:4d5c331c-2d1f-481b-a958-080182561712 nodeName:}" failed. No retries permitted until 2026-07-24 16:43:23.26204602 +0000 UTC m=+10.021854998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret") pod "global-pull-secret-syncer-jmx8q" (UID: "4d5c331c-2d1f-481b-a958-080182561712") : object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:19.362192 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:19.362152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:19.362393 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.362319    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 16:43:19.362393 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.362351    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 16:43:19.362393 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.362364    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q24vn for pod openshift-network-diagnostics/network-check-target-chhhw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:19.362566 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.362421    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn podName:9d026564-c995-470c-8186-7afe435c2b9f nodeName:}" failed. No retries permitted until 2026-07-24 16:43:23.362403333 +0000 UTC m=+10.122212326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q24vn" (UniqueName: "kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn") pod "network-check-target-chhhw" (UID: "9d026564-c995-470c-8186-7afe435c2b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:19.626071 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:19.625987    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:19.626543 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.626086    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:19.626543 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:19.626398    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:19.626543 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:19.626470    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:20.624212 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:20.624178    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:20.624402 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:20.624307    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:21.624587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:21.624551    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:21.625037 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:21.624692    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:21.625037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:21.624558    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:21.625155 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:21.625067    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:22.624532 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:22.624495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:22.624727 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:22.624651    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:23.296160 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:23.295904    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:23.296160 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:23.295971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:23.296160 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.296070    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:23.296160 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.296107    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:23.296160 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.296151    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret podName:4d5c331c-2d1f-481b-a958-080182561712 nodeName:}" failed. No retries permitted until 2026-07-24 16:43:31.296132722 +0000 UTC m=+18.055941702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret") pod "global-pull-secret-syncer-jmx8q" (UID: "4d5c331c-2d1f-481b-a958-080182561712") : object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:23.296160 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.296172    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:43:31.296162365 +0000 UTC m=+18.055971347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:23.397377 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:23.397315    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:23.397555 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.397505    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 16:43:23.397555 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.397528    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 16:43:23.397555 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.397540    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q24vn for pod openshift-network-diagnostics/network-check-target-chhhw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:23.397697 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.397603    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn podName:9d026564-c995-470c-8186-7afe435c2b9f nodeName:}" failed. No retries permitted until 2026-07-24 16:43:31.397583417 +0000 UTC m=+18.157392414 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q24vn" (UniqueName: "kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn") pod "network-check-target-chhhw" (UID: "9d026564-c995-470c-8186-7afe435c2b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:23.628696 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:23.628659    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:23.629120 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.628769    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:23.629120 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:23.628809    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:23.629120 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:23.628906    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:24.624272 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:24.624088    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:24.624272 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:24.624233    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:25.627126 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:25.624185    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:25.627126 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:25.624353    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:25.627126 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:25.624733    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:25.627126 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:25.624815    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:26.624067 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:26.624031    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:26.624233 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:26.624171    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:27.628602 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:27.628577    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:27.628996 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:27.628700    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:27.628996 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:27.628770    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:27.628996 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:27.628847    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:28.624027 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:28.623991    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:28.624224 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:28.624128    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:29.624919 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:29.624880    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:29.625429 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:29.624928    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:29.625429 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:29.625030    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:29.625539 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:29.625438    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:30.623970 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:30.623933    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:30.624198 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:30.624072    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:31.362130 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:31.362096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:31.362644 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:31.362171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:31.362644 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.362269    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:31.362644 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.362353    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:43:47.362319992 +0000 UTC m=+34.122128968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:31.362644 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.362271    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:31.362644 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.362408    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret podName:4d5c331c-2d1f-481b-a958-080182561712 nodeName:}" failed. No retries permitted until 2026-07-24 16:43:47.362396286 +0000 UTC m=+34.122205264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret") pod "global-pull-secret-syncer-jmx8q" (UID: "4d5c331c-2d1f-481b-a958-080182561712") : object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:31.463220 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:31.463180    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:31.463482 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.463362    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 16:43:31.463482 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.463387    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 16:43:31.463482 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.463400    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q24vn for pod openshift-network-diagnostics/network-check-target-chhhw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:31.463482 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.463465    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn podName:9d026564-c995-470c-8186-7afe435c2b9f nodeName:}" failed. No retries permitted until 2026-07-24 16:43:47.463446383 +0000 UTC m=+34.223255384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q24vn" (UniqueName: "kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn") pod "network-check-target-chhhw" (UID: "9d026564-c995-470c-8186-7afe435c2b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:31.626927 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:31.626849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:31.627089 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.626980    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:31.627089 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:31.627031    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:31.627194 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:31.627130    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:32.623879 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:32.623845    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:32.624354 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:32.623972    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:33.625529 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:33.625493    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:33.625999 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:33.625619    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:33.625999 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:33.625677    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:33.625999 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:33.625795    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:34.624792 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:34.624754    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:34.624990 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:34.624898    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:35.624350 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.624096    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:35.624798 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.624135    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:35.624798 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:35.624495    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:35.624798 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:35.624548    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:35.742000 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.741766    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2mvp5" event={"ID":"e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec","Type":"ContainerStarted","Data":"68aedac3cc8f653d1b01ad89909c545193f18458fbb24ee49e0566cb2ddf7020"}
Jul 24 16:43:35.743204 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.743180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8774l" event={"ID":"c1de7e8c-5014-4551-8158-d1c2101f616a","Type":"ContainerStarted","Data":"4a7b914dfd78ac7ca549186bd4c4d5e58dbc08c6fde8b8b0d97ed0a41b6299c0"}
Jul 24 16:43:35.744471 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.744451    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zld5x" event={"ID":"03ab4509-d158-4689-93c5-10571b0062f7","Type":"ContainerStarted","Data":"bd64ed922dd3d10f853133bdbb8edfba9efb88a53cf5fe50298e642f1bef1cad"}
Jul 24 16:43:35.745793 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.745764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sx8db" event={"ID":"9b3b84e6-8b95-41de-89fe-b68b7158ebec","Type":"ContainerStarted","Data":"9803fa6f9d99065a5e9a53514f54edee60ae91a74c72f4e8f00a2d6ab1afd670"}
Jul 24 16:43:35.747210 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.747191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b" event={"ID":"74780675-2b53-4d56-9b16-2d1f7e718afa","Type":"ContainerStarted","Data":"c6e44ace635b9167cc8b9ec7f4fcc810104c2b700470648317012bb5a24f37d1"}
Jul 24 16:43:35.748591 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.748570    2569 generic.go:356] "Generic (PLEG): container finished" podID="44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43" containerID="6fcbcfc77a38aef4a1d42466934107dce00b4c8d7e34833b36be4273acd86391" exitCode=0
Jul 24 16:43:35.748689 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.748637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerDied","Data":"6fcbcfc77a38aef4a1d42466934107dce00b4c8d7e34833b36be4273acd86391"}
Jul 24 16:43:35.750191 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.750101    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ptrlq" event={"ID":"25878a83-561c-45f2-9dc2-96d20c354f1e","Type":"ContainerStarted","Data":"ec82f6d5e587b6c514fe9c5889eb35b20bc6c1f46999682fade37ff4090b3b0f"}
Jul 24 16:43:35.751732 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.751660    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"d2b4fc7137800d26686b7ba9a148583feed4be58f887f0b1bed59ec878476322"}
Jul 24 16:43:35.757036 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.756999    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2mvp5" podStartSLOduration=4.241101734 podStartE2EDuration="22.756986998s" totalImagesPullingTime="18.515885264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:35.756554998 +0000 UTC m=+22.516363997" watchObservedRunningTime="2026-07-24 16:43:35.756986998 +0000 UTC m=+22.516795996"
Jul 24 16:43:35.757121 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.757062    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-204.ec2.internal" podStartSLOduration=21.757058081 podStartE2EDuration="21.757058081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:18.721934512 +0000 UTC m=+5.481743515" watchObservedRunningTime="2026-07-24 16:43:35.757058081 +0000 UTC m=+22.516867081"
Jul 24 16:43:35.773731 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.773691    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zld5x" podStartSLOduration=4.172146839 podStartE2EDuration="22.773678528s" totalImagesPullingTime="18.601531689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:35.773141338 +0000 UTC m=+22.532950336" watchObservedRunningTime="2026-07-24 16:43:35.773678528 +0000 UTC m=+22.533487528"
Jul 24 16:43:35.788607 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.788561    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sx8db" podStartSLOduration=8.358630279 podStartE2EDuration="22.788545911s" totalImagesPullingTime="14.429915632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:35.788375422 +0000 UTC m=+22.548184421" watchObservedRunningTime="2026-07-24 16:43:35.788545911 +0000 UTC m=+22.548354913"
Jul 24 16:43:35.808439 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.808397    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ptrlq" podStartSLOduration=4.243945961 podStartE2EDuration="22.808381581s" totalImagesPullingTime="18.56443562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:35.808268178 +0000 UTC m=+22.568077177" watchObservedRunningTime="2026-07-24 16:43:35.808381581 +0000 UTC m=+22.568190579"
Jul 24 16:43:35.840828 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:35.840787    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8774l" podStartSLOduration=4.324951345 podStartE2EDuration="22.840772792s" totalImagesPullingTime="18.515821447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:35.84045643 +0000 UTC m=+22.600265429" watchObservedRunningTime="2026-07-24 16:43:35.840772792 +0000 UTC m=+22.600581791"
Jul 24 16:43:36.624792 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.624765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:36.625450 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:36.624871    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:36.683513 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.683396    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 16:43:36.756069 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.756035    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b" event={"ID":"74780675-2b53-4d56-9b16-2d1f7e718afa","Type":"ContainerStarted","Data":"84a574f95c69c491d570770743fee86d40210ae681f461eb6f0cda320d18d33d"}
Jul 24 16:43:36.757725 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.757694    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wsjn9" event={"ID":"56bf2e0a-2c83-4387-8641-dab90c57c69f","Type":"ContainerStarted","Data":"e49e104dcbf30962abd6665283a6f604910e471f0458605251ed4996f0ec408a"}
Jul 24 16:43:36.761020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.760998    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"906058a4b8851a1567cd290352187235497e911b1d884c5e19224dacb8213287"}
Jul 24 16:43:36.761151 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.761060    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"69647bc43127f8560b8b3092d5208a2cda9b329eb6268749c02a188a26a201db"}
Jul 24 16:43:36.761151 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.761109    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"b4ce851a9c3906a8aea88fe06e6e53ee9bf2d26ba061f871ed5de8f3345bb8d6"}
Jul 24 16:43:36.761247 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.761154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"6a13a26c64cc51aa531d60eea1546e2e582f3c7e63a928657a29eca027853eb5"}
Jul 24 16:43:36.761247 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.761203    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"dad2aae8fb6a2a0cb873d3684836107979dbd10d74e36754167f1f53a1493e10"}
Jul 24 16:43:36.771252 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:36.771204    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wsjn9" podStartSLOduration=5.250050541 podStartE2EDuration="23.771191682s" totalImagesPullingTime="18.521141141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:36.771180594 +0000 UTC m=+23.530989594" watchObservedRunningTime="2026-07-24 16:43:36.771191682 +0000 UTC m=+23.531000682"
Jul 24 16:43:37.569415 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:37.569374    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:37.570106 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:37.570075    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:37.584360 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:37.584183    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T16:43:36.68341802Z","UUID":"907b0955-2a9f-4ddf-8cd1-6e17f648d432","Handler":null,"Name":"","Endpoint":""}
Jul 24 16:43:37.585968 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:37.585945    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 16:43:37.586079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:37.585986    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 16:43:37.624520 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:37.624493    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:37.624520 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:37.624512    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:37.624763 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:37.624610    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:37.624818 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:37.624760    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:38.624151 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:38.624119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:38.624462 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:38.624222    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:38.770393 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:38.770297    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b" event={"ID":"74780675-2b53-4d56-9b16-2d1f7e718afa","Type":"ContainerStarted","Data":"b2e4c53cb8d3005bb0e84938248157e8b237c9aa5d4a3a09425e438e306700af"}
Jul 24 16:43:38.774158 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:38.774126    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"8f208721e1d35e46da3e285a3c76067f0511a9d1d768a065400ed54b80185999"}
Jul 24 16:43:38.774158 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:38.774147    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 16:43:38.789039 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:38.788988    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tp98b" podStartSLOduration=5.079690996 podStartE2EDuration="25.788975511s" totalImagesPullingTime="20.709284515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:38.788816528 +0000 UTC m=+25.548625529" watchObservedRunningTime="2026-07-24 16:43:38.788975511 +0000 UTC m=+25.548784511"
Jul 24 16:43:39.624195 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:39.624158    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:39.624390 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:39.624164    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:39.624390 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:39.624294    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:39.624390 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:39.624379    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:40.624659 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.624626    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:40.625315 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:40.624741    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:40.780718 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.780637    2569 generic.go:356] "Generic (PLEG): container finished" podID="44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43" containerID="620e393c8d2364bdcbf9c46546b136f307768bb3fe77adc24fb56b0164c0ae58" exitCode=0
Jul 24 16:43:40.780859 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.780725    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerDied","Data":"620e393c8d2364bdcbf9c46546b136f307768bb3fe77adc24fb56b0164c0ae58"}
Jul 24 16:43:40.787454 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.787427    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" event={"ID":"81d06610-c1f7-4523-9af8-7691ca687a7e","Type":"ContainerStarted","Data":"996ac2b6380feabd7ad634a2e7f96f53731095409b3dd061635bdcfaea9f82df"}
Jul 24 16:43:40.787685 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.787665    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:40.787781 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.787740    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:40.803228 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.803209    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:40.825175 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:40.825132    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx" podStartSLOduration=8.886575524 podStartE2EDuration="27.825118162s" totalImagesPullingTime="18.938542638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:40.824861344 +0000 UTC m=+27.584670353" watchObservedRunningTime="2026-07-24 16:43:40.825118162 +0000 UTC m=+27.584927161"
Jul 24 16:43:41.627607 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:41.627411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:41.628065 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:41.627411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:41.628065 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:41.627684    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:41.628065 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:41.627801    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:41.789665 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:41.789639    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:41.805594 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:41.805570    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:43:42.396371 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.396314    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-chhhw"]
Jul 24 16:43:42.396570 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.396508    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:42.396648 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:42.396622    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:42.397124 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.397098    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jmx8q"]
Jul 24 16:43:42.397224 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.397211    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:42.397342 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:42.397303    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:42.398301 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.398277    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5mppg"]
Jul 24 16:43:42.398425 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.398393    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:42.398501 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:42.398481    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:42.793383 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.793353    2569 generic.go:356] "Generic (PLEG): container finished" podID="44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43" containerID="ae58de3042074bf0479e00f9555821c039c251844618560d7487c969d15f3e53" exitCode=0
Jul 24 16:43:42.793800 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:42.793430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerDied","Data":"ae58de3042074bf0479e00f9555821c039c251844618560d7487c969d15f3e53"}
Jul 24 16:43:43.625101 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:43.625073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:43.625254 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:43.625151    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:43.625254 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:43.625232    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:43.625376 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:43.625353    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:44.342600 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:44.342561    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:44.343024 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:44.342755    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 16:43:44.343280 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:44.343259    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sx8db"
Jul 24 16:43:44.624008 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:44.623793    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:44.624144 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:44.624039    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:44.800158 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:44.800123    2569 generic.go:356] "Generic (PLEG): container finished" podID="44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43" containerID="78719c531ededf6b0b85b174ee05ee4225f8e0b57113d8c29144cb1919461e28" exitCode=0
Jul 24 16:43:44.800307 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:44.800169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerDied","Data":"78719c531ededf6b0b85b174ee05ee4225f8e0b57113d8c29144cb1919461e28"}
Jul 24 16:43:45.624684 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:45.624659    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:45.625268 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:45.624668    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:45.625268 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:45.624781    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:45.625268 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:45.624842    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:46.624659 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:46.624629    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:46.624826 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:46.624736    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:43:47.385200 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:47.385152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:47.385200 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:47.385212    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:47.385496 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.385314    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:47.385496 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.385320    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:47.385496 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.385382    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:44:19.38536705 +0000 UTC m=+66.145176027 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 16:43:47.385496 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.385412    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret podName:4d5c331c-2d1f-481b-a958-080182561712 nodeName:}" failed. No retries permitted until 2026-07-24 16:44:19.38539479 +0000 UTC m=+66.145203767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret") pod "global-pull-secret-syncer-jmx8q" (UID: "4d5c331c-2d1f-481b-a958-080182561712") : object "kube-system"/"original-pull-secret" not registered
Jul 24 16:43:47.486387 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:47.486347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:47.486574 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.486514    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 16:43:47.486574 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.486535    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 16:43:47.486574 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.486548    2569 projected.go:196] Error preparing data for projected volume kube-api-access-q24vn for pod openshift-network-diagnostics/network-check-target-chhhw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:47.486690 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.486612    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn podName:9d026564-c995-470c-8186-7afe435c2b9f nodeName:}" failed. No retries permitted until 2026-07-24 16:44:19.486594199 +0000 UTC m=+66.246403179 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q24vn" (UniqueName: "kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn") pod "network-check-target-chhhw" (UID: "9d026564-c995-470c-8186-7afe435c2b9f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 16:43:47.624619 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:47.624578    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:47.624619 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:47.624620    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:47.624841 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.624728    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-chhhw" podUID="9d026564-c995-470c-8186-7afe435c2b9f"
Jul 24 16:43:47.624841 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:47.624824    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jmx8q" podUID="4d5c331c-2d1f-481b-a958-080182561712"
Jul 24 16:43:48.058629 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.058601    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-204.ec2.internal" event="NodeReady"
Jul 24 16:43:48.058806 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.058752    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 16:43:48.107912 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.107456    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-57k6f"]
Jul 24 16:43:48.112985 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.112950    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8ppx4"]
Jul 24 16:43:48.113148 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.113098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.115676 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.115648    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 16:43:48.115799 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.115706    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 16:43:48.115799 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.115747    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6r7g7\""
Jul 24 16:43:48.116815 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.116796    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.119320 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.118981    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 16:43:48.119320 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.119027    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 16:43:48.119320 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.119186    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 16:43:48.119320 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.119268    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nk7wj\""
Jul 24 16:43:48.122361 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.121869    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-57k6f"]
Jul 24 16:43:48.123631 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.123600    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8ppx4"]
Jul 24 16:43:48.193651 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.193565    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plfx7\" (UniqueName: \"kubernetes.io/projected/88ca08d7-89d0-4507-86c3-8231fa6e116a-kube-api-access-plfx7\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.193651 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.193606    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88ca08d7-89d0-4507-86c3-8231fa6e116a-config-volume\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.193651 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.193627    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.193651 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.193650    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88ca08d7-89d0-4507-86c3-8231fa6e116a-tmp-dir\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.294562 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.294527    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plfx7\" (UniqueName: \"kubernetes.io/projected/88ca08d7-89d0-4507-86c3-8231fa6e116a-kube-api-access-plfx7\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.294562 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.294558    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88ca08d7-89d0-4507-86c3-8231fa6e116a-config-volume\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.294788 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.294580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.294788 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.294606    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88ca08d7-89d0-4507-86c3-8231fa6e116a-tmp-dir\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.294788 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:48.294715    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:43:48.294788 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.294754    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/900c12f2-0612-4eb9-a0b4-5f93b9c8e097-cert\") pod \"ingress-canary-8ppx4\" (UID: \"900c12f2-0612-4eb9-a0b4-5f93b9c8e097\") " pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.294978 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:48.294795    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:43:48.79477328 +0000 UTC m=+35.554582261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:43:48.294978 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.294859    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rxcj\" (UniqueName: \"kubernetes.io/projected/900c12f2-0612-4eb9-a0b4-5f93b9c8e097-kube-api-access-4rxcj\") pod \"ingress-canary-8ppx4\" (UID: \"900c12f2-0612-4eb9-a0b4-5f93b9c8e097\") " pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.295089 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.295024    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88ca08d7-89d0-4507-86c3-8231fa6e116a-tmp-dir\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.295363 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.295344    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88ca08d7-89d0-4507-86c3-8231fa6e116a-config-volume\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.305618 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.305593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plfx7\" (UniqueName: \"kubernetes.io/projected/88ca08d7-89d0-4507-86c3-8231fa6e116a-kube-api-access-plfx7\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.395724 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.395684    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/900c12f2-0612-4eb9-a0b4-5f93b9c8e097-cert\") pod \"ingress-canary-8ppx4\" (UID: \"900c12f2-0612-4eb9-a0b4-5f93b9c8e097\") " pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.395724 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.395728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rxcj\" (UniqueName: \"kubernetes.io/projected/900c12f2-0612-4eb9-a0b4-5f93b9c8e097-kube-api-access-4rxcj\") pod \"ingress-canary-8ppx4\" (UID: \"900c12f2-0612-4eb9-a0b4-5f93b9c8e097\") " pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.398243 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.398217    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/900c12f2-0612-4eb9-a0b4-5f93b9c8e097-cert\") pod \"ingress-canary-8ppx4\" (UID: \"900c12f2-0612-4eb9-a0b4-5f93b9c8e097\") " pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.404424 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.404394    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rxcj\" (UniqueName: \"kubernetes.io/projected/900c12f2-0612-4eb9-a0b4-5f93b9c8e097-kube-api-access-4rxcj\") pod \"ingress-canary-8ppx4\" (UID: \"900c12f2-0612-4eb9-a0b4-5f93b9c8e097\") " pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.433427 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.433392    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8ppx4"
Jul 24 16:43:48.624009 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.623973    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:43:48.627946 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.627922    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 16:43:48.628638 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.627993    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6gtf2\""
Jul 24 16:43:48.800453 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:48.800421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:48.800613 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:48.800590    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:43:48.800682 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:48.800659    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:43:49.800642538 +0000 UTC m=+36.560451519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:43:49.269774 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.269606    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8ppx4"]
Jul 24 16:43:49.275938 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:43:49.274029    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900c12f2_0612_4eb9_a0b4_5f93b9c8e097.slice/crio-fb0dc1383f0f164b55205b28acc94c98255f941e273d1887ff61fe7c7621dff2 WatchSource:0}: Error finding container fb0dc1383f0f164b55205b28acc94c98255f941e273d1887ff61fe7c7621dff2: Status 404 returned error can't find the container with id fb0dc1383f0f164b55205b28acc94c98255f941e273d1887ff61fe7c7621dff2
Jul 24 16:43:49.624132 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.624038    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:43:49.624284 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.624041    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:43:49.628229 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.628207    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 16:43:49.629165 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.629145    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nn52b\""
Jul 24 16:43:49.629266 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.629155    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 16:43:49.629266 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.629180    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 16:43:49.811376 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.811347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:49.811557 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:49.811460    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:43:49.811557 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:49.811543    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:43:51.811522089 +0000 UTC m=+38.571331128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:43:49.813609 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.813583    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8ppx4" event={"ID":"900c12f2-0612-4eb9-a0b4-5f93b9c8e097","Type":"ContainerStarted","Data":"fb0dc1383f0f164b55205b28acc94c98255f941e273d1887ff61fe7c7621dff2"}
Jul 24 16:43:49.816577 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.816551    2569 generic.go:356] "Generic (PLEG): container finished" podID="44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43" containerID="59fae03645bf77d802ec8cf1a8609e71e55971b781d6ef10e429b806c9e615cd" exitCode=0
Jul 24 16:43:49.816725 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:49.816614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerDied","Data":"59fae03645bf77d802ec8cf1a8609e71e55971b781d6ef10e429b806c9e615cd"}
Jul 24 16:43:50.823116 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:50.823084    2569 generic.go:356] "Generic (PLEG): container finished" podID="44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43" containerID="f9854e10ca3dbc4fad08cbbfd93e065ceb25e41566c422f5c60c4612264deaef" exitCode=0
Jul 24 16:43:50.823638 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:50.823143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerDied","Data":"f9854e10ca3dbc4fad08cbbfd93e065ceb25e41566c422f5c60c4612264deaef"}
Jul 24 16:43:51.830163 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:51.828036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:51.830163 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:51.828462    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:43:51.830163 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:51.828555    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:43:55.828533783 +0000 UTC m=+42.588342760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:43:51.831935 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:51.831908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" event={"ID":"44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43","Type":"ContainerStarted","Data":"a3b162c5f7bdd7816c72ff021f3e7cb7a1552ee6e61a16c4a682057cae03a37c"}
Jul 24 16:43:51.833200 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:51.833178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8ppx4" event={"ID":"900c12f2-0612-4eb9-a0b4-5f93b9c8e097","Type":"ContainerStarted","Data":"f5da7e17db7e1a456e7ec9f1c96e06b26f5d552fc52fda29338b63e4215eb145"}
Jul 24 16:43:51.854248 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:51.854195    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8hzgt" podStartSLOduration=9.437285597 podStartE2EDuration="38.854181954s" totalImagesPullingTime="29.416896357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:51.852908354 +0000 UTC m=+38.612717352" watchObservedRunningTime="2026-07-24 16:43:51.854181954 +0000 UTC m=+38.613990949"
Jul 24 16:43:51.869299 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:51.869258    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8ppx4" podStartSLOduration=1.693759392 podStartE2EDuration="3.869245875s" totalImagesPullingTime="2.175486483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:43:51.8684429 +0000 UTC m=+38.628251899" watchObservedRunningTime="2026-07-24 16:43:51.869245875 +0000 UTC m=+38.629054874"
Jul 24 16:43:55.856894 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:43:55.856860    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:43:55.857532 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:55.856994    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:43:55.857532 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:43:55.857060    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:44:03.857041331 +0000 UTC m=+50.616850322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:44:03.911862 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:03.911825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:44:03.912245 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:03.911949    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:44:03.912245 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:03.912033    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:44:19.912019203 +0000 UTC m=+66.671828180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:44:13.816573 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:13.816540    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wprhx"
Jul 24 16:44:19.424659 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.424614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:44:19.425143 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.424761    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:44:19.426985 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.426963    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 16:44:19.427081 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.427065    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 16:44:19.434824 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:19.434793    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 16:44:19.434947 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:19.434917    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:45:23.434894882 +0000 UTC m=+130.194703874 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : secret "metrics-daemon-secret" not found
Jul 24 16:44:19.438569 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.438549    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4d5c331c-2d1f-481b-a958-080182561712-original-pull-secret\") pod \"global-pull-secret-syncer-jmx8q\" (UID: \"4d5c331c-2d1f-481b-a958-080182561712\") " pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:44:19.525787 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.525754    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:44:19.528402 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.528387    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 16:44:19.539265 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.539243    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 16:44:19.549741 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.549718    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q24vn\" (UniqueName: \"kubernetes.io/projected/9d026564-c995-470c-8186-7afe435c2b9f-kube-api-access-q24vn\") pod \"network-check-target-chhhw\" (UID: \"9d026564-c995-470c-8186-7afe435c2b9f\") " pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:44:19.635267 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.635239    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nn52b\""
Jul 24 16:44:19.638173 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.638159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jmx8q"
Jul 24 16:44:19.643864 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.643840    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:44:19.766861 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.766833    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jmx8q"]
Jul 24 16:44:19.783409 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.782267    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-chhhw"]
Jul 24 16:44:19.785911 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:44:19.785878    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d026564_c995_470c_8186_7afe435c2b9f.slice/crio-59791513c9e234173f25955370fdfaf5eeffc329134e5df4e8aa9ada670bb23c WatchSource:0}: Error finding container 59791513c9e234173f25955370fdfaf5eeffc329134e5df4e8aa9ada670bb23c: Status 404 returned error can't find the container with id 59791513c9e234173f25955370fdfaf5eeffc329134e5df4e8aa9ada670bb23c
Jul 24 16:44:19.900624 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.900583    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-chhhw" event={"ID":"9d026564-c995-470c-8186-7afe435c2b9f","Type":"ContainerStarted","Data":"59791513c9e234173f25955370fdfaf5eeffc329134e5df4e8aa9ada670bb23c"}
Jul 24 16:44:19.901604 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.901579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jmx8q" event={"ID":"4d5c331c-2d1f-481b-a958-080182561712","Type":"ContainerStarted","Data":"9699774314b6a0ae5033c080a598001e065b4f3e6868146631c412352e591a3d"}
Jul 24 16:44:19.929176 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:19.929143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:44:19.929297 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:19.929283    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:44:19.929386 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:19.929373    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:44:51.929357086 +0000 UTC m=+98.689166062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:44:24.915548 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:24.915516    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-chhhw" event={"ID":"9d026564-c995-470c-8186-7afe435c2b9f","Type":"ContainerStarted","Data":"9f5eb34683f77b7ad71227c6c4c46ab78e7eae6242f01b952e130e5004232202"}
Jul 24 16:44:24.916050 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:24.915640    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:44:24.916861 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:24.916836    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jmx8q" event={"ID":"4d5c331c-2d1f-481b-a958-080182561712","Type":"ContainerStarted","Data":"a5cf320921d655e3be9566afc7664e7e0c2e9ade442239fedf1df1893cb5c91e"}
Jul 24 16:44:24.930739 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:24.930694    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-chhhw" podStartSLOduration=67.363719399 podStartE2EDuration="1m11.930681028s" totalImagesPullingTime="4.566961629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:44:24.929323206 +0000 UTC m=+71.689132204" watchObservedRunningTime="2026-07-24 16:44:24.930681028 +0000 UTC m=+71.690490027"
Jul 24 16:44:24.942323 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:24.942282    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jmx8q" podStartSLOduration=67.354544982 podStartE2EDuration="1m11.942270904s" totalImagesPullingTime="4.587725922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:44:24.941854804 +0000 UTC m=+71.701663816" watchObservedRunningTime="2026-07-24 16:44:24.942270904 +0000 UTC m=+71.702079902"
Jul 24 16:44:31.036994 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.036957    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"]
Jul 24 16:44:31.039861 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.039844    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.041730 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.041709    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 16:44:31.042182 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.042160    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 16:44:31.043165 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.043145    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 16:44:31.043269 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.043179    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 16:44:31.049917 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.049896    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"]
Jul 24 16:44:31.108120 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.108076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a585ed74-7260-4bab-bc91-9f15a96092be-tmp\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.108120 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.108117    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b47hb\" (UniqueName: \"kubernetes.io/projected/a585ed74-7260-4bab-bc91-9f15a96092be-kube-api-access-b47hb\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.108350 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.108179    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a585ed74-7260-4bab-bc91-9f15a96092be-klusterlet-config\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.209389 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.209353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a585ed74-7260-4bab-bc91-9f15a96092be-klusterlet-config\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.209541 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.209430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a585ed74-7260-4bab-bc91-9f15a96092be-tmp\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.209541 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.209451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b47hb\" (UniqueName: \"kubernetes.io/projected/a585ed74-7260-4bab-bc91-9f15a96092be-kube-api-access-b47hb\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.209870 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.209849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a585ed74-7260-4bab-bc91-9f15a96092be-tmp\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.211808 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.211791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a585ed74-7260-4bab-bc91-9f15a96092be-klusterlet-config\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.218035 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.218012    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b47hb\" (UniqueName: \"kubernetes.io/projected/a585ed74-7260-4bab-bc91-9f15a96092be-kube-api-access-b47hb\") pod \"klusterlet-addon-workmgr-6968df74cd-dtkjk\" (UID: \"a585ed74-7260-4bab-bc91-9f15a96092be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.350057 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.350008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:31.470995 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.470965    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"]
Jul 24 16:44:31.474154 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:44:31.474125    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda585ed74_7260_4bab_bc91_9f15a96092be.slice/crio-fd7464cbd8de85de99caf9bce718971e9c8fe087e31e432066a9035a7a23b0d9 WatchSource:0}: Error finding container fd7464cbd8de85de99caf9bce718971e9c8fe087e31e432066a9035a7a23b0d9: Status 404 returned error can't find the container with id fd7464cbd8de85de99caf9bce718971e9c8fe087e31e432066a9035a7a23b0d9
Jul 24 16:44:31.935367 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:31.935320    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk" event={"ID":"a585ed74-7260-4bab-bc91-9f15a96092be","Type":"ContainerStarted","Data":"fd7464cbd8de85de99caf9bce718971e9c8fe087e31e432066a9035a7a23b0d9"}
Jul 24 16:44:35.946436 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:35.946284    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk" event={"ID":"a585ed74-7260-4bab-bc91-9f15a96092be","Type":"ContainerStarted","Data":"e7042699b3a2a6175093bee2c1487c295312ea29a76549edb29ee43827892f1b"}
Jul 24 16:44:35.947068 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:35.946538    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:35.948204 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:35.948186    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:44:35.963746 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:35.963695    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk" podStartSLOduration=1.447632665 podStartE2EDuration="4.963681792s" totalImagesPullingTime="3.516049127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:44:35.962891116 +0000 UTC m=+82.722700129" watchObservedRunningTime="2026-07-24 16:44:35.963681792 +0000 UTC m=+82.723490790"
Jul 24 16:44:51.952493 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:51.952451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:44:51.952952 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:51.952563    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:44:51.952952 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:44:51.952614    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:45:55.952601123 +0000 UTC m=+162.712410100 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:44:55.922314 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:44:55.922281    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-chhhw"
Jul 24 16:45:23.463561 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:45:23.463509    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:45:23.464043 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:45:23.463660    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 16:45:23.464043 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:45:23.463742    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:47:25.463725934 +0000 UTC m=+252.223534911 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : secret "metrics-daemon-secret" not found
Jul 24 16:45:51.127241 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:45:51.127201    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-57k6f" podUID="88ca08d7-89d0-4507-86c3-8231fa6e116a"
Jul 24 16:45:51.633746 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:45:51.633707    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:45:52.124438 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:45:52.124411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57k6f"
Jul 24 16:45:55.982959 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:45:55.982918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:45:55.983379 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:45:55.983069    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 16:45:55.983379 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:45:55.983157    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls podName:88ca08d7-89d0-4507-86c3-8231fa6e116a nodeName:}" failed. No retries permitted until 2026-07-24 16:47:57.983141287 +0000 UTC m=+284.742950265 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls") pod "dns-default-57k6f" (UID: "88ca08d7-89d0-4507-86c3-8231fa6e116a") : secret "dns-default-metrics-tls" not found
Jul 24 16:46:02.624344 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:02.624290    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:46:35.226219 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:35.226180    2569 generic.go:356] "Generic (PLEG): container finished" podID="a585ed74-7260-4bab-bc91-9f15a96092be" containerID="e7042699b3a2a6175093bee2c1487c295312ea29a76549edb29ee43827892f1b" exitCode=1
Jul 24 16:46:35.226734 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:35.226241    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk" event={"ID":"a585ed74-7260-4bab-bc91-9f15a96092be","Type":"ContainerDied","Data":"e7042699b3a2a6175093bee2c1487c295312ea29a76549edb29ee43827892f1b"}
Jul 24 16:46:35.226734 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:35.226677    2569 scope.go:117] "RemoveContainer" containerID="e7042699b3a2a6175093bee2c1487c295312ea29a76549edb29ee43827892f1b"
Jul 24 16:46:35.947252 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:35.947206    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:46:36.230281 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:36.230188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk" event={"ID":"a585ed74-7260-4bab-bc91-9f15a96092be","Type":"ContainerStarted","Data":"782e867ccbe6c478fb5451fed75e903afa6642b2a1ff9cbca2d1aa892336112b"}
Jul 24 16:46:36.230675 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:36.230404    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:46:36.230977 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:46:36.230964    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6968df74cd-dtkjk"
Jul 24 16:47:25.067398 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.067363    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"]
Jul 24 16:47:25.070144 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.070127    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bbb7dbb8d-6jskq"]
Jul 24 16:47:25.070261 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.070243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.072700 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.072675    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 24 16:47:25.072816 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.072756    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.072974 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.072952    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 24 16:47:25.073129 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.073112    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ztl9n\""
Jul 24 16:47:25.073196 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.073178    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 24 16:47:25.073251 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.073219    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 24 16:47:25.075565 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.075528    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4mtk4\""
Jul 24 16:47:25.075897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.075884    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 16:47:25.076397 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.076384    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 16:47:25.076476 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.076455    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 16:47:25.080865 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.080846    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 16:47:25.084119 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.084100    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"]
Jul 24 16:47:25.087117 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.087097    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bbb7dbb8d-6jskq"]
Jul 24 16:47:25.159132 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159095    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-certificates\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.159132 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-installation-pull-secrets\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.159380 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgddq\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-kube-api-access-mgddq\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.159380 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-trusted-ca\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.159380 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d473c729-80ec-4439-9407-df3b25da6534-serving-cert\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.159380 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/d473c729-80ec-4439-9407-df3b25da6534-kube-api-access-k6f4b\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.159567 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159413    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.159567 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-bound-sa-token\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.159567 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d473c729-80ec-4439-9407-df3b25da6534-config\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.159567 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-image-registry-private-configuration\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.159690 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.159578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c34e8de-fd01-40a2-81d3-52ece5d86c90-ca-trust-extracted\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.171278 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.171255    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"]
Jul 24 16:47:25.174031 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.174016    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.176088 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.176072    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 24 16:47:25.176772 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.176752    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4rttr\""
Jul 24 16:47:25.177558 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.177525    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 16:47:25.177652 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.177579    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 24 16:47:25.177652 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.177612    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 16:47:25.191519 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.191488    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"]
Jul 24 16:47:25.260360 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d473c729-80ec-4439-9407-df3b25da6534-config\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.260360 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-image-registry-private-configuration\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.260579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260388    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c34e8de-fd01-40a2-81d3-52ece5d86c90-ca-trust-extracted\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.260579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260416    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.260579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260443    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chpkf\" (UniqueName: \"kubernetes.io/projected/aa3d1740-2d73-48c8-bc56-66c995d90073-kube-api-access-chpkf\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.260579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-certificates\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.260579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260497    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-installation-pull-secrets\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.260579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260562    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aa3d1740-2d73-48c8-bc56-66c995d90073-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgddq\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-kube-api-access-mgddq\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260666    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-trusted-ca\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d473c729-80ec-4439-9407-df3b25da6534-serving-cert\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/d473c729-80ec-4439-9407-df3b25da6534-kube-api-access-k6f4b\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-bound-sa-token\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260872    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c34e8de-fd01-40a2-81d3-52ece5d86c90-ca-trust-extracted\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.261013 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.260997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d473c729-80ec-4439-9407-df3b25da6534-config\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.261424 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.261151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-certificates\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.261424 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.261159    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 16:47:25.261424 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.261197    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bbb7dbb8d-6jskq: secret "image-registry-tls" not found
Jul 24 16:47:25.261424 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.261268    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls podName:4c34e8de-fd01-40a2-81d3-52ece5d86c90 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:25.761248415 +0000 UTC m=+252.521057407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls") pod "image-registry-bbb7dbb8d-6jskq" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90") : secret "image-registry-tls" not found
Jul 24 16:47:25.261765 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.261749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-trusted-ca\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.262950 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.262918    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d473c729-80ec-4439-9407-df3b25da6534-serving-cert\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.263230 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.263212    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-installation-pull-secrets\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.263306 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.263261    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-image-registry-private-configuration\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.270813 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.270786    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgddq\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-kube-api-access-mgddq\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.271447 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.271424    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-bound-sa-token\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.271877 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.271854    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/d473c729-80ec-4439-9407-df3b25da6534-kube-api-access-k6f4b\") pod \"service-ca-operator-5d6c75f84f-mn2vm\" (UID: \"d473c729-80ec-4439-9407-df3b25da6534\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.361296 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.361221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.361296 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.361252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chpkf\" (UniqueName: \"kubernetes.io/projected/aa3d1740-2d73-48c8-bc56-66c995d90073-kube-api-access-chpkf\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.361296 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.361276    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aa3d1740-2d73-48c8-bc56-66c995d90073-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.361535 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.361376    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:25.361535 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.361442    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls podName:aa3d1740-2d73-48c8-bc56-66c995d90073 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:25.861427291 +0000 UTC m=+252.621236268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cbt4j" (UID: "aa3d1740-2d73-48c8-bc56-66c995d90073") : secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:25.361940 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.361924    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aa3d1740-2d73-48c8-bc56-66c995d90073-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.370064 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.370037    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chpkf\" (UniqueName: \"kubernetes.io/projected/aa3d1740-2d73-48c8-bc56-66c995d90073-kube-api-access-chpkf\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.380719 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.380700    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"
Jul 24 16:47:25.498585 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.498554    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm"]
Jul 24 16:47:25.503438 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:25.503408    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd473c729_80ec_4439_9407_df3b25da6534.slice/crio-eb9321adf0ff5dcdaf05dd72e8e6e43bf0253462ed0a5c43326b78ed4df8348c WatchSource:0}: Error finding container eb9321adf0ff5dcdaf05dd72e8e6e43bf0253462ed0a5c43326b78ed4df8348c: Status 404 returned error can't find the container with id eb9321adf0ff5dcdaf05dd72e8e6e43bf0253462ed0a5c43326b78ed4df8348c
Jul 24 16:47:25.563186 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.563149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:47:25.563373 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.563296    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 16:47:25.563426 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.563393    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs podName:12520acc-f016-4ab9-bc6e-3efd878c185b nodeName:}" failed. No retries permitted until 2026-07-24 16:49:27.563376653 +0000 UTC m=+374.323185631 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs") pod "network-metrics-daemon-5mppg" (UID: "12520acc-f016-4ab9-bc6e-3efd878c185b") : secret "metrics-daemon-secret" not found
Jul 24 16:47:25.765005 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.764893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:25.765177 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.765043    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 16:47:25.765177 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.765062    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bbb7dbb8d-6jskq: secret "image-registry-tls" not found
Jul 24 16:47:25.765177 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.765112    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls podName:4c34e8de-fd01-40a2-81d3-52ece5d86c90 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:26.765097603 +0000 UTC m=+253.524906580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls") pod "image-registry-bbb7dbb8d-6jskq" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90") : secret "image-registry-tls" not found
Jul 24 16:47:25.865411 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:25.865372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:25.865577 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.865519    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:25.865625 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:25.865583    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls podName:aa3d1740-2d73-48c8-bc56-66c995d90073 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:26.86556815 +0000 UTC m=+253.625377127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cbt4j" (UID: "aa3d1740-2d73-48c8-bc56-66c995d90073") : secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:26.354020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:26.353981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm" event={"ID":"d473c729-80ec-4439-9407-df3b25da6534","Type":"ContainerStarted","Data":"eb9321adf0ff5dcdaf05dd72e8e6e43bf0253462ed0a5c43326b78ed4df8348c"}
Jul 24 16:47:26.771413 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:26.771309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:26.771564 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:26.771471    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 16:47:26.771564 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:26.771492    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bbb7dbb8d-6jskq: secret "image-registry-tls" not found
Jul 24 16:47:26.771564 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:26.771543    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls podName:4c34e8de-fd01-40a2-81d3-52ece5d86c90 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:28.771529066 +0000 UTC m=+255.531338044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls") pod "image-registry-bbb7dbb8d-6jskq" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90") : secret "image-registry-tls" not found
Jul 24 16:47:26.872421 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:26.872381    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:26.872603 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:26.872531    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:26.872603 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:26.872598    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls podName:aa3d1740-2d73-48c8-bc56-66c995d90073 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:28.872583099 +0000 UTC m=+255.632392076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cbt4j" (UID: "aa3d1740-2d73-48c8-bc56-66c995d90073") : secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:28.361699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:28.361663    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm" event={"ID":"d473c729-80ec-4439-9407-df3b25da6534","Type":"ContainerStarted","Data":"f6a11eb2587c3e0ce459bdf3bc64e05371355d1dc42eabd91345e24eefdb4605"}
Jul 24 16:47:28.378061 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:28.378008    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm" podStartSLOduration=1.205543639 podStartE2EDuration="3.37799393s" totalImagesPullingTime="2.172450291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:47:28.37706166 +0000 UTC m=+255.136870660" watchObservedRunningTime="2026-07-24 16:47:28.37799393 +0000 UTC m=+255.137802946"
Jul 24 16:47:28.787090 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:28.787005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:28.787228 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:28.787158    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 16:47:28.787228 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:28.787174    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bbb7dbb8d-6jskq: secret "image-registry-tls" not found
Jul 24 16:47:28.787316 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:28.787229    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls podName:4c34e8de-fd01-40a2-81d3-52ece5d86c90 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:32.787213076 +0000 UTC m=+259.547022061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls") pod "image-registry-bbb7dbb8d-6jskq" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90") : secret "image-registry-tls" not found
Jul 24 16:47:28.888073 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:28.888028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:28.888242 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:28.888171    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:28.888242 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:28.888233    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls podName:aa3d1740-2d73-48c8-bc56-66c995d90073 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:32.888218278 +0000 UTC m=+259.648027256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cbt4j" (UID: "aa3d1740-2d73-48c8-bc56-66c995d90073") : secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:30.025497 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.025463    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g72bh"]
Jul 24 16:47:30.028605 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.028586    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.030612 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.030589    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 16:47:30.030741 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.030624    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 16:47:30.030938 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.030923    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 16:47:30.031537 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.031516    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k88vr\""
Jul 24 16:47:30.031644 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.031541    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 16:47:30.040655 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.040633    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g72bh"]
Jul 24 16:47:30.097385 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.097346    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-crio-socket\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.097541 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.097400    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.097541 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.097418    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.097541 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.097440    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-data-volume\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.097541 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.097467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m5rx\" (UniqueName: \"kubernetes.io/projected/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-api-access-5m5rx\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.198674    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.198787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.198841    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.198858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-data-volume\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.198900    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m5rx\" (UniqueName: \"kubernetes.io/projected/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-api-access-5m5rx\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.198919    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:30.698896632 +0000 UTC m=+257.458705610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.199045    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-crio-socket\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.199164    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-crio-socket\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.199248    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.199302    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:30.699287669 +0000 UTC m=+257.459096663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : configmap "kube-rbac-proxy" not found
Jul 24 16:47:30.201988 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.199619    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-data-volume\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.217683 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.217653    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m5rx\" (UniqueName: \"kubernetes.io/projected/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-api-access-5m5rx\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.225081 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.225057    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"]
Jul 24 16:47:30.228063 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.228045    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"
Jul 24 16:47:30.230343 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.230309    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 16:47:30.230430 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.230417    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 16:47:30.230476 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.230461    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zx9jt\""
Jul 24 16:47:30.241610 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.241590    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"]
Jul 24 16:47:30.300495 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.300414    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcz4h\" (UniqueName: \"kubernetes.io/projected/9e26248c-ee8d-4c05-ae8d-98a85820e121-kube-api-access-tcz4h\") pod \"migrator-6865c9967-8tms4\" (UID: \"9e26248c-ee8d-4c05-ae8d-98a85820e121\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"
Jul 24 16:47:30.401753 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.401714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcz4h\" (UniqueName: \"kubernetes.io/projected/9e26248c-ee8d-4c05-ae8d-98a85820e121-kube-api-access-tcz4h\") pod \"migrator-6865c9967-8tms4\" (UID: \"9e26248c-ee8d-4c05-ae8d-98a85820e121\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"
Jul 24 16:47:30.409880 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.409846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcz4h\" (UniqueName: \"kubernetes.io/projected/9e26248c-ee8d-4c05-ae8d-98a85820e121-kube-api-access-tcz4h\") pod \"migrator-6865c9967-8tms4\" (UID: \"9e26248c-ee8d-4c05-ae8d-98a85820e121\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"
Jul 24 16:47:30.537042 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.536998    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"
Jul 24 16:47:30.662722 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.662683    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4"]
Jul 24 16:47:30.665700 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:30.665675    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e26248c_ee8d_4c05_ae8d_98a85820e121.slice/crio-deedd579cad8b85a180f07fe1259a47774b4469ee790ecbaed2a82dec00a9823 WatchSource:0}: Error finding container deedd579cad8b85a180f07fe1259a47774b4469ee790ecbaed2a82dec00a9823: Status 404 returned error can't find the container with id deedd579cad8b85a180f07fe1259a47774b4469ee790ecbaed2a82dec00a9823
Jul 24 16:47:30.703816 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.703771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.703950 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:30.703821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:30.703950 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.703896    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 16:47:30.704033 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.703952    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:31.703934856 +0000 UTC m=+258.463743835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : configmap "kube-rbac-proxy" not found
Jul 24 16:47:30.704033 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.703994    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:30.704126 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:30.704052    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:31.704039466 +0000 UTC m=+258.463848443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:31.373706 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.373672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4" event={"ID":"9e26248c-ee8d-4c05-ae8d-98a85820e121","Type":"ContainerStarted","Data":"deedd579cad8b85a180f07fe1259a47774b4469ee790ecbaed2a82dec00a9823"}
Jul 24 16:47:31.461076 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.461041    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-wjr6l"]
Jul 24 16:47:31.464121 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.464096    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.466637 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.466610    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 24 16:47:31.466760 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.466618    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 24 16:47:31.467572 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.467533    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 24 16:47:31.467729 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.467583    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-49zgv\""
Jul 24 16:47:31.467729 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.467613    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 24 16:47:31.471712 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.471666    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-wjr6l"]
Jul 24 16:47:31.611095 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.611057    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6dbb9896-347f-492b-baa2-9fe0418c09b0-signing-key\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.611278 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.611108    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6dbb9896-347f-492b-baa2-9fe0418c09b0-signing-cabundle\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.611278 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.611236    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p4hs\" (UniqueName: \"kubernetes.io/projected/6dbb9896-347f-492b-baa2-9fe0418c09b0-kube-api-access-6p4hs\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.711972 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.711908    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:31.711972 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.711944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:31.712169 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.711988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p4hs\" (UniqueName: \"kubernetes.io/projected/6dbb9896-347f-492b-baa2-9fe0418c09b0-kube-api-access-6p4hs\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.712169 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.712026    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6dbb9896-347f-492b-baa2-9fe0418c09b0-signing-key\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.712169 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:31.712038    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 16:47:31.712169 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.712045    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6dbb9896-347f-492b-baa2-9fe0418c09b0-signing-cabundle\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.712169 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:31.712103    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:33.712083966 +0000 UTC m=+260.471892942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : configmap "kube-rbac-proxy" not found
Jul 24 16:47:31.712169 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:31.712127    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:31.712169 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:31.712173    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:33.712160169 +0000 UTC m=+260.471969146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:31.712763 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.712743    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6dbb9896-347f-492b-baa2-9fe0418c09b0-signing-cabundle\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.714215 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.714199    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6dbb9896-347f-492b-baa2-9fe0418c09b0-signing-key\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.723780 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.723758    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p4hs\" (UniqueName: \"kubernetes.io/projected/6dbb9896-347f-492b-baa2-9fe0418c09b0-kube-api-access-6p4hs\") pod \"service-ca-658d7b7c78-wjr6l\" (UID: \"6dbb9896-347f-492b-baa2-9fe0418c09b0\") " pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.768828 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.768805    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8774l_c1de7e8c-5014-4551-8158-d1c2101f616a/dns-node-resolver/0.log"
Jul 24 16:47:31.775011 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.774993    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l"
Jul 24 16:47:31.908034 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:31.908001    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-wjr6l"]
Jul 24 16:47:31.911657 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:31.911632    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dbb9896_347f_492b_baa2_9fe0418c09b0.slice/crio-e6bd16d631d88cab5724bf51e514daea099cf7ecdecb24c9af43df3d0514d87f WatchSource:0}: Error finding container e6bd16d631d88cab5724bf51e514daea099cf7ecdecb24c9af43df3d0514d87f: Status 404 returned error can't find the container with id e6bd16d631d88cab5724bf51e514daea099cf7ecdecb24c9af43df3d0514d87f
Jul 24 16:47:32.378094 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.378064    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l" event={"ID":"6dbb9896-347f-492b-baa2-9fe0418c09b0","Type":"ContainerStarted","Data":"4451877ab96766df53ba519042578b617792e51a7c47e20d46c528c301d24a54"}
Jul 24 16:47:32.378594 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.378108    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l" event={"ID":"6dbb9896-347f-492b-baa2-9fe0418c09b0","Type":"ContainerStarted","Data":"e6bd16d631d88cab5724bf51e514daea099cf7ecdecb24c9af43df3d0514d87f"}
Jul 24 16:47:32.379776 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.379752    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4" event={"ID":"9e26248c-ee8d-4c05-ae8d-98a85820e121","Type":"ContainerStarted","Data":"1c3f7461f02b55f798b302fd77d881a5028d7ae1e9846ed13a5a2c228c9de3c6"}
Jul 24 16:47:32.379898 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.379790    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4" event={"ID":"9e26248c-ee8d-4c05-ae8d-98a85820e121","Type":"ContainerStarted","Data":"a195e00f74b776b04e6b6f133db17c2cabb4af1ff71edfd972f1c1f71576b7f8"}
Jul 24 16:47:32.397763 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.397716    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-wjr6l" podStartSLOduration=1.397700663 podStartE2EDuration="1.397700663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:47:32.3965631 +0000 UTC m=+259.156372099" watchObservedRunningTime="2026-07-24 16:47:32.397700663 +0000 UTC m=+259.157509692"
Jul 24 16:47:32.414072 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.414025    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8tms4" podStartSLOduration=1.3384924790000001 podStartE2EDuration="2.414008598s" totalImagesPullingTime="1.075516119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:47:32.413063855 +0000 UTC m=+259.172872854" watchObservedRunningTime="2026-07-24 16:47:32.414008598 +0000 UTC m=+259.173817619"
Jul 24 16:47:32.769949 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.769870    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2mvp5_e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec/node-ca/0.log"
Jul 24 16:47:32.822241 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.822201    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:32.822455 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:32.822362    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 16:47:32.822455 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:32.822378    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bbb7dbb8d-6jskq: secret "image-registry-tls" not found
Jul 24 16:47:32.822455 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:32.822434    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls podName:4c34e8de-fd01-40a2-81d3-52ece5d86c90 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:40.822418546 +0000 UTC m=+267.582227523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls") pod "image-registry-bbb7dbb8d-6jskq" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90") : secret "image-registry-tls" not found
Jul 24 16:47:32.922926 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:32.922885    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:32.923095 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:32.923060    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:32.923152 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:32.923140    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls podName:aa3d1740-2d73-48c8-bc56-66c995d90073 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:40.923119581 +0000 UTC m=+267.682928563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-cbt4j" (UID: "aa3d1740-2d73-48c8-bc56-66c995d90073") : secret "cluster-monitoring-operator-tls" not found
Jul 24 16:47:33.569774 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:33.569746    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8ppx4_900c12f2-0612-4eb9-a0b4-5f93b9c8e097/serve-healthcheck-canary/0.log"
Jul 24 16:47:33.728761 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:33.728701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:33.728942 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:33.728760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:33.728942 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:33.728839    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 16:47:33.728942 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:33.728929    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:37.728909169 +0000 UTC m=+264.488718151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : configmap "kube-rbac-proxy" not found
Jul 24 16:47:33.729132 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:33.728957    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:33.729132 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:33.729018    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:37.728999946 +0000 UTC m=+264.488808926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:37.763762 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:37.763669    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:37.763762 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:37.763720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:37.763762 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:37.763752    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 16:47:37.764310 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:37.763813    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:45.763794993 +0000 UTC m=+272.523603974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : configmap "kube-rbac-proxy" not found
Jul 24 16:47:37.764310 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:37.764233    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:37.764310 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:37.764295    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls podName:3b57bd0b-8952-4fcd-821c-db4ccfb03bbc nodeName:}" failed. No retries permitted until 2026-07-24 16:47:45.764278977 +0000 UTC m=+272.524087962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g72bh" (UID: "3b57bd0b-8952-4fcd-821c-db4ccfb03bbc") : secret "insights-runtime-extractor-tls" not found
Jul 24 16:47:40.889400 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:40.889361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:40.891725 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:40.891703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"image-registry-bbb7dbb8d-6jskq\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") " pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:40.985892 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:40.985859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:40.990080 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:40.990053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:40.992592 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:40.992564    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aa3d1740-2d73-48c8-bc56-66c995d90073-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-cbt4j\" (UID: \"aa3d1740-2d73-48c8-bc56-66c995d90073\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:41.083006 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.082957    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"
Jul 24 16:47:41.118930 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.118894    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bbb7dbb8d-6jskq"]
Jul 24 16:47:41.121930 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:41.121896    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c34e8de_fd01_40a2_81d3_52ece5d86c90.slice/crio-5dcaf328c5e702bf5ba079b58d86e1e1b3cec6b71788fed5b2736bba4a0b3d42 WatchSource:0}: Error finding container 5dcaf328c5e702bf5ba079b58d86e1e1b3cec6b71788fed5b2736bba4a0b3d42: Status 404 returned error can't find the container with id 5dcaf328c5e702bf5ba079b58d86e1e1b3cec6b71788fed5b2736bba4a0b3d42
Jul 24 16:47:41.215719 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.215692    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j"]
Jul 24 16:47:41.218816 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:41.218790    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3d1740_2d73_48c8_bc56_66c995d90073.slice/crio-26c161554c8719f1a1fd58c8984980fe49ff7f1a621435d79291b5a316c36f08 WatchSource:0}: Error finding container 26c161554c8719f1a1fd58c8984980fe49ff7f1a621435d79291b5a316c36f08: Status 404 returned error can't find the container with id 26c161554c8719f1a1fd58c8984980fe49ff7f1a621435d79291b5a316c36f08
Jul 24 16:47:41.409146 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.409059    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j" event={"ID":"aa3d1740-2d73-48c8-bc56-66c995d90073","Type":"ContainerStarted","Data":"26c161554c8719f1a1fd58c8984980fe49ff7f1a621435d79291b5a316c36f08"}
Jul 24 16:47:41.410671 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.410647    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq" event={"ID":"4c34e8de-fd01-40a2-81d3-52ece5d86c90","Type":"ContainerStarted","Data":"8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7"}
Jul 24 16:47:41.410778 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.410687    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq" event={"ID":"4c34e8de-fd01-40a2-81d3-52ece5d86c90","Type":"ContainerStarted","Data":"5dcaf328c5e702bf5ba079b58d86e1e1b3cec6b71788fed5b2736bba4a0b3d42"}
Jul 24 16:47:41.410850 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.410789    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:47:41.429776 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:41.429730    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq" podStartSLOduration=16.429713557 podStartE2EDuration="16.429713557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:47:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:47:41.428739264 +0000 UTC m=+268.188548267" watchObservedRunningTime="2026-07-24 16:47:41.429713557 +0000 UTC m=+268.189522553"
Jul 24 16:47:43.419895 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:43.419804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j" event={"ID":"aa3d1740-2d73-48c8-bc56-66c995d90073","Type":"ContainerStarted","Data":"791b8a10b720b6511acc20037c086137446748bd1bd6258c096065b8700ac5e1"}
Jul 24 16:47:43.443319 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:43.443256    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-cbt4j" podStartSLOduration=16.556702642 podStartE2EDuration="18.443238497s" totalImagesPullingTime="1.886535855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:47:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:47:43.441644223 +0000 UTC m=+270.201453251" watchObservedRunningTime="2026-07-24 16:47:43.443238497 +0000 UTC m=+270.203047495"
Jul 24 16:47:45.832740 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:45.832694    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:45.832740 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:45.832738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:45.833238 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:45.833216    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:45.835056 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:45.835013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3b57bd0b-8952-4fcd-821c-db4ccfb03bbc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g72bh\" (UID: \"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc\") " pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:45.937425 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:45.937393    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g72bh"
Jul 24 16:47:46.060200 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:46.060171    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g72bh"]
Jul 24 16:47:46.063278 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:46.063249    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b57bd0b_8952_4fcd_821c_db4ccfb03bbc.slice/crio-6e26d8083dd26cc50cae1a73deb29f9d8c88321887462f010f0eb188167a0596 WatchSource:0}: Error finding container 6e26d8083dd26cc50cae1a73deb29f9d8c88321887462f010f0eb188167a0596: Status 404 returned error can't find the container with id 6e26d8083dd26cc50cae1a73deb29f9d8c88321887462f010f0eb188167a0596
Jul 24 16:47:46.430841 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:46.430801    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g72bh" event={"ID":"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc","Type":"ContainerStarted","Data":"6e26d8083dd26cc50cae1a73deb29f9d8c88321887462f010f0eb188167a0596"}
Jul 24 16:47:50.524643 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.524602    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bbb7dbb8d-6jskq"]
Jul 24 16:47:50.578137 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.578083    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-676787f6b8-s598c"]
Jul 24 16:47:50.581859 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.581823    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.602512 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.602172    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676787f6b8-s598c"]
Jul 24 16:47:50.669318 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-bound-sa-token\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.669506 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669366    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/061a4548-0c73-4be6-8148-69da8f83c662-image-registry-private-configuration\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.669506 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/061a4548-0c73-4be6-8148-69da8f83c662-trusted-ca\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.669506 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669503    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-registry-tls\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.669665 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669530    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/061a4548-0c73-4be6-8148-69da8f83c662-installation-pull-secrets\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.669665 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669606    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/061a4548-0c73-4be6-8148-69da8f83c662-registry-certificates\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.669665 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669645    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7g4v\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-kube-api-access-p7g4v\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.669898 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.669688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/061a4548-0c73-4be6-8148-69da8f83c662-ca-trust-extracted\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770207 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/061a4548-0c73-4be6-8148-69da8f83c662-trusted-ca\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770426 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770216    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-registry-tls\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770426 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/061a4548-0c73-4be6-8148-69da8f83c662-installation-pull-secrets\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770426 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/061a4548-0c73-4be6-8148-69da8f83c662-registry-certificates\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770610 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7g4v\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-kube-api-access-p7g4v\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770685 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/061a4548-0c73-4be6-8148-69da8f83c662-ca-trust-extracted\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770752 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770707    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-bound-sa-token\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.770811 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.770779    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/061a4548-0c73-4be6-8148-69da8f83c662-image-registry-private-configuration\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.771126 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.771068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/061a4548-0c73-4be6-8148-69da8f83c662-ca-trust-extracted\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.771344 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.771304    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/061a4548-0c73-4be6-8148-69da8f83c662-registry-certificates\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.771444 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.771415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/061a4548-0c73-4be6-8148-69da8f83c662-trusted-ca\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.773220 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.773194    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/061a4548-0c73-4be6-8148-69da8f83c662-installation-pull-secrets\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.773348 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.773308    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-registry-tls\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.773577 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.773553    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/061a4548-0c73-4be6-8148-69da8f83c662-image-registry-private-configuration\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.781872 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.781816    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-bound-sa-token\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.782632 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.782614    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7g4v\" (UniqueName: \"kubernetes.io/projected/061a4548-0c73-4be6-8148-69da8f83c662-kube-api-access-p7g4v\") pod \"image-registry-676787f6b8-s598c\" (UID: \"061a4548-0c73-4be6-8148-69da8f83c662\") " pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:50.893839 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:50.893819    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:51.023455 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:51.023426    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-676787f6b8-s598c"]
Jul 24 16:47:51.027108 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:51.027078    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061a4548_0c73_4be6_8148_69da8f83c662.slice/crio-6659e8ad0bedeef93da252d6bb2824abe3bbcea7b2b29ef6f9838f3935772374 WatchSource:0}: Error finding container 6659e8ad0bedeef93da252d6bb2824abe3bbcea7b2b29ef6f9838f3935772374: Status 404 returned error can't find the container with id 6659e8ad0bedeef93da252d6bb2824abe3bbcea7b2b29ef6f9838f3935772374
Jul 24 16:47:51.452202 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:51.451466    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676787f6b8-s598c" event={"ID":"061a4548-0c73-4be6-8148-69da8f83c662","Type":"ContainerStarted","Data":"02c3ea6552da0859bc8b6537ba255375c76608d9340b4725f9ae54b1ac466329"}
Jul 24 16:47:51.452202 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:51.451526    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-676787f6b8-s598c" event={"ID":"061a4548-0c73-4be6-8148-69da8f83c662","Type":"ContainerStarted","Data":"6659e8ad0bedeef93da252d6bb2824abe3bbcea7b2b29ef6f9838f3935772374"}
Jul 24 16:47:51.452202 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:51.452155    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:47:51.454690 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:51.454370    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g72bh" event={"ID":"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc","Type":"ContainerStarted","Data":"2c34041769700bcef19630f9db2337f809728136b04702b42d1413c7f92d139c"}
Jul 24 16:47:51.470860 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:51.470809    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-676787f6b8-s598c" podStartSLOduration=1.470789179 podStartE2EDuration="1.470789179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:47:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:47:51.469516476 +0000 UTC m=+278.229325470" watchObservedRunningTime="2026-07-24 16:47:51.470789179 +0000 UTC m=+278.230598178"
Jul 24 16:47:54.465997 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:54.465962    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g72bh" event={"ID":"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc","Type":"ContainerStarted","Data":"582f61e9aa668f4ab0d548ce64bef50bd764f8172383ca293e1949304e29fa40"}
Jul 24 16:47:55.124909 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:55.124865    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-57k6f" podUID="88ca08d7-89d0-4507-86c3-8231fa6e116a"
Jul 24 16:47:55.469472 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:55.469391    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57k6f"
Jul 24 16:47:58.042030 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.041990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:47:58.044219 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.044185    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88ca08d7-89d0-4507-86c3-8231fa6e116a-metrics-tls\") pod \"dns-default-57k6f\" (UID: \"88ca08d7-89d0-4507-86c3-8231fa6e116a\") " pod="openshift-dns/dns-default-57k6f"
Jul 24 16:47:58.172272 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.172241    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6r7g7\""
Jul 24 16:47:58.180224 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.180202    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-57k6f"
Jul 24 16:47:58.303954 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.303930    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-57k6f"]
Jul 24 16:47:58.306177 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:58.306147    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88ca08d7_89d0_4507_86c3_8231fa6e116a.slice/crio-dfbb93793fc30ce8cab6ec6dd0c5a6d5c3792f6da88739d65b148c1ae576b037 WatchSource:0}: Error finding container dfbb93793fc30ce8cab6ec6dd0c5a6d5c3792f6da88739d65b148c1ae576b037: Status 404 returned error can't find the container with id dfbb93793fc30ce8cab6ec6dd0c5a6d5c3792f6da88739d65b148c1ae576b037
Jul 24 16:47:58.481403 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.481298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g72bh" event={"ID":"3b57bd0b-8952-4fcd-821c-db4ccfb03bbc","Type":"ContainerStarted","Data":"14055fa6e1b5db687f7473e126b8c709b997019b529135620cb975c7265a77b7"}
Jul 24 16:47:58.482465 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.482443    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57k6f" event={"ID":"88ca08d7-89d0-4507-86c3-8231fa6e116a","Type":"ContainerStarted","Data":"dfbb93793fc30ce8cab6ec6dd0c5a6d5c3792f6da88739d65b148c1ae576b037"}
Jul 24 16:47:58.498821 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:58.498776    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g72bh" podStartSLOduration=17.876137485 podStartE2EDuration="29.498761303s" totalImagesPullingTime="11.622623818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:47:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:47:58.49753893 +0000 UTC m=+285.257347930" watchObservedRunningTime="2026-07-24 16:47:58.498761303 +0000 UTC m=+285.258570307"
Jul 24 16:47:59.061079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.060910    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x5mq4"]
Jul 24 16:47:59.064612 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.064490    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.066943 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.066919    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 16:47:59.067157 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.067136    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 16:47:59.067233 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.067136    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9scxg\""
Jul 24 16:47:59.067731 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.067711    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 16:47:59.067934 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.067918    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 16:47:59.151762 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62611e08-01c1-4139-8308-b6236b703357-metrics-client-ca\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.151762 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6jnm\" (UniqueName: \"kubernetes.io/projected/62611e08-01c1-4139-8308-b6236b703357-kube-api-access-w6jnm\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.151980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151787    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-tls\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.151980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62611e08-01c1-4139-8308-b6236b703357-node-exporter-textfile\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.151980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151838    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-node-exporter-wtmp\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.151980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151862    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-root\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.151980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151892    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.151980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62611e08-01c1-4139-8308-b6236b703357-node-exporter-accelerators-collector-config\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.152242 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.151995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-sys\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.253155 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62611e08-01c1-4139-8308-b6236b703357-metrics-client-ca\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.253363 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253164    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6jnm\" (UniqueName: \"kubernetes.io/projected/62611e08-01c1-4139-8308-b6236b703357-kube-api-access-w6jnm\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.253363 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253183    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-tls\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62611e08-01c1-4139-8308-b6236b703357-node-exporter-textfile\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-node-exporter-wtmp\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-root\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.253976    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-node-exporter-wtmp\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254608 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.254053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62611e08-01c1-4139-8308-b6236b703357-node-exporter-accelerators-collector-config\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254608 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.254102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62611e08-01c1-4139-8308-b6236b703357-node-exporter-textfile\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.254608 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.254060    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-root\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.257604 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.254769    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62611e08-01c1-4139-8308-b6236b703357-metrics-client-ca\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.257604 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.255054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-sys\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.257604 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.255180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62611e08-01c1-4139-8308-b6236b703357-sys\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.257604 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:59.255272    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 16:47:59.257604 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:47:59.255348    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-tls podName:62611e08-01c1-4139-8308-b6236b703357 nodeName:}" failed. No retries permitted until 2026-07-24 16:47:59.755316136 +0000 UTC m=+286.515125129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-tls") pod "node-exporter-x5mq4" (UID: "62611e08-01c1-4139-8308-b6236b703357") : secret "node-exporter-tls" not found
Jul 24 16:47:59.257604 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.255893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62611e08-01c1-4139-8308-b6236b703357-node-exporter-accelerators-collector-config\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.260110 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.258192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.263214 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.263188    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6jnm\" (UniqueName: \"kubernetes.io/projected/62611e08-01c1-4139-8308-b6236b703357-kube-api-access-w6jnm\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.760221 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.760182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-tls\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.762932 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.762900    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62611e08-01c1-4139-8308-b6236b703357-node-exporter-tls\") pod \"node-exporter-x5mq4\" (UID: \"62611e08-01c1-4139-8308-b6236b703357\") " pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.978821 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:47:59.978791    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x5mq4"
Jul 24 16:47:59.990640 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:47:59.990610    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62611e08_01c1_4139_8308_b6236b703357.slice/crio-48a2f9127b5338021efe04303597da62c5d1e1189a89878d2e3dafe79e7a2cf9 WatchSource:0}: Error finding container 48a2f9127b5338021efe04303597da62c5d1e1189a89878d2e3dafe79e7a2cf9: Status 404 returned error can't find the container with id 48a2f9127b5338021efe04303597da62c5d1e1189a89878d2e3dafe79e7a2cf9
Jul 24 16:48:00.490897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:00.490843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mq4" event={"ID":"62611e08-01c1-4139-8308-b6236b703357","Type":"ContainerStarted","Data":"48a2f9127b5338021efe04303597da62c5d1e1189a89878d2e3dafe79e7a2cf9"}
Jul 24 16:48:00.492834 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:00.492796    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57k6f" event={"ID":"88ca08d7-89d0-4507-86c3-8231fa6e116a","Type":"ContainerStarted","Data":"8742401fefdfd72bf7310da8dfc5c53fd5a1c37e03b3336f3a65d1ca6056b209"}
Jul 24 16:48:00.492959 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:00.492847    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-57k6f" event={"ID":"88ca08d7-89d0-4507-86c3-8231fa6e116a","Type":"ContainerStarted","Data":"e417ebba7f3329859fed9b36b51fa473b57c3572b43ecec407fbcddc276feb39"}
Jul 24 16:48:00.492959 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:00.492945    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-57k6f"
Jul 24 16:48:00.509695 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:00.509649    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-57k6f" podStartSLOduration=250.786595261 podStartE2EDuration="4m12.50963259s" totalImagesPullingTime="1.723037329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:48:00.508843351 +0000 UTC m=+287.268652351" watchObservedRunningTime="2026-07-24 16:48:00.50963259 +0000 UTC m=+287.269441585"
Jul 24 16:48:00.531040 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:00.531006    2569 patch_prober.go:28] interesting pod/image-registry-bbb7dbb8d-6jskq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 16:48:00.531172 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:00.531060    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq" podUID="4c34e8de-fd01-40a2-81d3-52ece5d86c90" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 16:48:01.014773 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.014737    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7478c6548b-8lmx8"]
Jul 24 16:48:01.018264 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.018248    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.020831 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.020797    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 16:48:01.020942 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.020861    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 16:48:01.020942 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.020890    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-18o4912nnj8kf\""
Jul 24 16:48:01.020942 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.020917    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 16:48:01.021179 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.021162    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-f49xp\""
Jul 24 16:48:01.021237 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.021219    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 16:48:01.021401 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.021234    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 16:48:01.029888 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.029868    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7478c6548b-8lmx8"]
Jul 24 16:48:01.072701 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.072808 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072712    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-grpc-tls\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.072808 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.072808 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072792    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klznl\" (UniqueName: \"kubernetes.io/projected/01e535b2-bb78-482b-b3bf-0db3de302090-kube-api-access-klznl\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.072926 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072843    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-tls\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.072926 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.072926 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072894    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01e535b2-bb78-482b-b3bf-0db3de302090-metrics-client-ca\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.072926 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.072916    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173561 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173617    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-grpc-tls\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173660    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klznl\" (UniqueName: \"kubernetes.io/projected/01e535b2-bb78-482b-b3bf-0db3de302090-kube-api-access-klznl\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173869 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-tls\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173869 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.173869 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.173749    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01e535b2-bb78-482b-b3bf-0db3de302090-metrics-client-ca\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.174559 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.174504    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01e535b2-bb78-482b-b3bf-0db3de302090-metrics-client-ca\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.176198 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.176171    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-grpc-tls\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.176311 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.176210    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.176405 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.176351    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.176466 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.176404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-tls\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.176532 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.176466    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.176621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.176601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/01e535b2-bb78-482b-b3bf-0db3de302090-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.181234 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.181216    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klznl\" (UniqueName: \"kubernetes.io/projected/01e535b2-bb78-482b-b3bf-0db3de302090-kube-api-access-klznl\") pod \"thanos-querier-7478c6548b-8lmx8\" (UID: \"01e535b2-bb78-482b-b3bf-0db3de302090\") " pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.329247 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.329206    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:01.484978 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.484950    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7478c6548b-8lmx8"]
Jul 24 16:48:01.486001 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:48:01.485975    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e535b2_bb78_482b_b3bf_0db3de302090.slice/crio-92b3c3bdceac3e5c007631ad76888e1726b4981f3646c7f356ddd054e4f47619 WatchSource:0}: Error finding container 92b3c3bdceac3e5c007631ad76888e1726b4981f3646c7f356ddd054e4f47619: Status 404 returned error can't find the container with id 92b3c3bdceac3e5c007631ad76888e1726b4981f3646c7f356ddd054e4f47619
Jul 24 16:48:01.497156 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.497112    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" event={"ID":"01e535b2-bb78-482b-b3bf-0db3de302090","Type":"ContainerStarted","Data":"92b3c3bdceac3e5c007631ad76888e1726b4981f3646c7f356ddd054e4f47619"}
Jul 24 16:48:01.498431 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.498408    2569 generic.go:356] "Generic (PLEG): container finished" podID="62611e08-01c1-4139-8308-b6236b703357" containerID="0d71c2307283360218352115976bbd107aa2e6a451f25043884b9b62b8425bd2" exitCode=0
Jul 24 16:48:01.498525 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:01.498439    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mq4" event={"ID":"62611e08-01c1-4139-8308-b6236b703357","Type":"ContainerDied","Data":"0d71c2307283360218352115976bbd107aa2e6a451f25043884b9b62b8425bd2"}
Jul 24 16:48:02.505530 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:02.505497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mq4" event={"ID":"62611e08-01c1-4139-8308-b6236b703357","Type":"ContainerStarted","Data":"72df123dec23f0db66a7629049d86ff38fcdf938e4e21d462c8e871f686960d9"}
Jul 24 16:48:02.506020 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:02.505560    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x5mq4" event={"ID":"62611e08-01c1-4139-8308-b6236b703357","Type":"ContainerStarted","Data":"d4112e4ebea2c302adb7439176015f6223c632b88994393fcebc72f8f93bee9a"}
Jul 24 16:48:02.525224 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:02.525167    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x5mq4" podStartSLOduration=2.436132461 podStartE2EDuration="3.525152236s" totalImagesPullingTime="1.025910253s" totalInitContainerRuntime="63.109522ms" isStatefulPod=true podCreationTimestamp="2026-07-24 16:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:48:02.524107689 +0000 UTC m=+289.283916688" watchObservedRunningTime="2026-07-24 16:48:02.525152236 +0000 UTC m=+289.284961236"
Jul 24 16:48:03.815220 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.815191    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"]
Jul 24 16:48:03.818780 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.818762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:03.820955 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.820914    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 16:48:03.821193 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.821017    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9m2v5\""
Jul 24 16:48:03.825430 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.825347    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"]
Jul 24 16:48:03.896215 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.896111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswk9\" (UniqueName: \"kubernetes.io/projected/22bcd108-fd34-4335-9435-99d389361e21-kube-api-access-dswk9\") pod \"monitoring-plugin-68b7fc9574-z7m96\" (UID: \"22bcd108-fd34-4335-9435-99d389361e21\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:03.896215 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.896161    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/22bcd108-fd34-4335-9435-99d389361e21-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-z7m96\" (UID: \"22bcd108-fd34-4335-9435-99d389361e21\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:03.997027 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.996987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dswk9\" (UniqueName: \"kubernetes.io/projected/22bcd108-fd34-4335-9435-99d389361e21-kube-api-access-dswk9\") pod \"monitoring-plugin-68b7fc9574-z7m96\" (UID: \"22bcd108-fd34-4335-9435-99d389361e21\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:03.997027 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.997024    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/22bcd108-fd34-4335-9435-99d389361e21-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-z7m96\" (UID: \"22bcd108-fd34-4335-9435-99d389361e21\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:03.999534 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:03.999510    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/22bcd108-fd34-4335-9435-99d389361e21-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-z7m96\" (UID: \"22bcd108-fd34-4335-9435-99d389361e21\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:04.004150 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:04.004113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswk9\" (UniqueName: \"kubernetes.io/projected/22bcd108-fd34-4335-9435-99d389361e21-kube-api-access-dswk9\") pod \"monitoring-plugin-68b7fc9574-z7m96\" (UID: \"22bcd108-fd34-4335-9435-99d389361e21\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:04.130411 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:04.130380    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:04.254863 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:04.254833    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"]
Jul 24 16:48:04.258068 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:48:04.258041    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22bcd108_fd34_4335_9435_99d389361e21.slice/crio-63056128fcd6ffff519a8ca21f9eba71da52acb9c25306c523ff27b11893f791 WatchSource:0}: Error finding container 63056128fcd6ffff519a8ca21f9eba71da52acb9c25306c523ff27b11893f791: Status 404 returned error can't find the container with id 63056128fcd6ffff519a8ca21f9eba71da52acb9c25306c523ff27b11893f791
Jul 24 16:48:04.515645 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:04.515556    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" event={"ID":"01e535b2-bb78-482b-b3bf-0db3de302090","Type":"ContainerStarted","Data":"20641195938b979c2c2b31d3472c31650fbbbfc7f627f9362755f34150654865"}
Jul 24 16:48:04.515645 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:04.515636    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" event={"ID":"01e535b2-bb78-482b-b3bf-0db3de302090","Type":"ContainerStarted","Data":"a859695c8efe0c0cdba6a813d390f462dbfbd4e9aedf605ee00f39bd9535cd69"}
Jul 24 16:48:04.515844 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:04.515675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" event={"ID":"01e535b2-bb78-482b-b3bf-0db3de302090","Type":"ContainerStarted","Data":"07cea37ab320d04887c21934b4f766099b5ab6eb53fe9552e2f7dc420b45236e"}
Jul 24 16:48:04.516812 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:04.516789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96" event={"ID":"22bcd108-fd34-4335-9435-99d389361e21","Type":"ContainerStarted","Data":"63056128fcd6ffff519a8ca21f9eba71da52acb9c25306c523ff27b11893f791"}
Jul 24 16:48:05.190693 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.190657    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:48:05.194390 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.194373    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.199721 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199694    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 16:48:05.199877 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199859    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bi021q3t25spa\""
Jul 24 16:48:05.199954 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199698    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 16:48:05.199954 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199936    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 16:48:05.199954 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199943    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 16:48:05.200103 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199848    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 16:48:05.200103 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199735    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 16:48:05.200103 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199816    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 16:48:05.200244 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.199758    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 16:48:05.200349 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.200317    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 16:48:05.200413 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.200338    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 16:48:05.200465 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.200442    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 16:48:05.200465 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.200401    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-48bt9\""
Jul 24 16:48:05.200561 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.200471    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 16:48:05.200610 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.200564    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 16:48:05.200750 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.200729    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 16:48:05.202226 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.202210    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 16:48:05.213598 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.213573    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:48:05.306530 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306706 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306706 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306576    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306706 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306642    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306706 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306706 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306697    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306934 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306934 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306784    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306934 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306814    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306934 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.306934 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.306989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.307005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.307024    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.307071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.307100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7qzw\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-kube-api-access-q7qzw\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.307125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.307392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.307148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408558 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408558 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408558 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408738 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408738 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408607    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408738 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408650    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408738 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408687    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408738 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408790    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.408984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7qzw\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-kube-api-access-q7qzw\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.409321 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.408993    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.409321 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.409016    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.409321 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.409077    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.409321 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.409126    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.409536 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.409442    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.409896 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.409628    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.410117 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.410093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.411536 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.410851    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.411536 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.411022    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.411694 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.411622    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.412093 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.411385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.413139 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.413115    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.413778 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.413726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.413891 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.413859    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.413973 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.413889    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.414323 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.414302    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.414513 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.414464    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.414619 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.414541    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.414684 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.414657    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.415127 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.415110    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.415409 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.415385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.415844 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.415822    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.416950 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.416919    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.419418 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.419395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7qzw\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-kube-api-access-q7qzw\") pod \"prometheus-k8s-0\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.508844 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.508748    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:05.525796 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.525759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" event={"ID":"01e535b2-bb78-482b-b3bf-0db3de302090","Type":"ContainerStarted","Data":"fc0d6f548ca8ffcb8b5beb57b4ac7f6f6f5a79732ab391b77a4701034cc170c4"}
Jul 24 16:48:05.525973 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.525840    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" event={"ID":"01e535b2-bb78-482b-b3bf-0db3de302090","Type":"ContainerStarted","Data":"58f49dc5a6d1f84d8c1c3ff2a3c5fdbd43af7319063a193a77baa1f729622a9c"}
Jul 24 16:48:05.525973 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.525883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" event={"ID":"01e535b2-bb78-482b-b3bf-0db3de302090","Type":"ContainerStarted","Data":"694ffa9b34b8ca08ed24d2b9dce083f4290d9bbca0f8a9cba133053a7d07e6ce"}
Jul 24 16:48:05.525973 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.525932    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:05.551019 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.550963    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8" podStartSLOduration=2.53300899 podStartE2EDuration="5.550944568s" totalImagesPullingTime="3.017935578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:48:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:48:05.549423398 +0000 UTC m=+292.309232398" watchObservedRunningTime="2026-07-24 16:48:05.550944568 +0000 UTC m=+292.310753597"
Jul 24 16:48:05.625766 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:48:05.625722    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5mppg" podUID="12520acc-f016-4ab9-bc6e-3efd878c185b"
Jul 24 16:48:05.785155 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:05.785131    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:48:05.787074 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:48:05.787039    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e271f6_f94d_459a_8c9e_75c393e7d4b3.slice/crio-66995050b78513b042c3a58f15bb3878515511c08ee1eda9369ace99670c7b8f WatchSource:0}: Error finding container 66995050b78513b042c3a58f15bb3878515511c08ee1eda9369ace99670c7b8f: Status 404 returned error can't find the container with id 66995050b78513b042c3a58f15bb3878515511c08ee1eda9369ace99670c7b8f
Jul 24 16:48:06.532397 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:06.532104    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96" event={"ID":"22bcd108-fd34-4335-9435-99d389361e21","Type":"ContainerStarted","Data":"3ce2229c6ebdec7d2c335a7a689a0bdc6706673937f270acd69bcc87f34b9296"}
Jul 24 16:48:06.532397 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:06.532318    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:06.533710 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:06.533675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerStarted","Data":"66995050b78513b042c3a58f15bb3878515511c08ee1eda9369ace99670c7b8f"}
Jul 24 16:48:06.537859 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:06.537811    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96"
Jul 24 16:48:06.548924 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:06.548877    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-z7m96" podStartSLOduration=2.113918756 podStartE2EDuration="3.548862692s" totalImagesPullingTime="1.434943936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:48:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:48:06.546720392 +0000 UTC m=+293.306529404" watchObservedRunningTime="2026-07-24 16:48:06.548862692 +0000 UTC m=+293.308671687"
Jul 24 16:48:07.538696 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:07.538611    2569 generic.go:356] "Generic (PLEG): container finished" podID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerID="f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3" exitCode=0
Jul 24 16:48:07.539034 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:07.538710    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3"}
Jul 24 16:48:10.501478 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:10.501449    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-57k6f"
Jul 24 16:48:10.530573 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:10.530539    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:48:11.539830 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.539802    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7478c6548b-8lmx8"
Jul 24 16:48:11.560048 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.560017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerStarted","Data":"e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402"}
Jul 24 16:48:11.560195 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.560085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerStarted","Data":"dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa"}
Jul 24 16:48:11.560195 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.560116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerStarted","Data":"a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140"}
Jul 24 16:48:11.560195 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.560146    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerStarted","Data":"e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6"}
Jul 24 16:48:11.560195 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.560173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerStarted","Data":"13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a"}
Jul 24 16:48:11.560324 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.560199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerStarted","Data":"87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5"}
Jul 24 16:48:11.591564 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:11.591511    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.058022954 podStartE2EDuration="6.591493754s" totalImagesPullingTime="4.51489756s" totalInitContainerRuntime="18.57324ms" isStatefulPod=false podCreationTimestamp="2026-07-24 16:48:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:48:11.589575425 +0000 UTC m=+298.349384449" watchObservedRunningTime="2026-07-24 16:48:11.591493754 +0000 UTC m=+298.351302752"
Jul 24 16:48:13.465275 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:13.465247    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-676787f6b8-s598c"
Jul 24 16:48:13.557441 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:13.557417    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 16:48:15.509240 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.509202    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:48:15.545150 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.545092    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq" podUID="4c34e8de-fd01-40a2-81d3-52ece5d86c90" containerName="registry" containerID="cri-o://8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7" gracePeriod=30
Jul 24 16:48:15.790634 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.790609    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:48:15.906849 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.906812    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-trusted-ca\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907024 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.906865    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-bound-sa-token\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907024 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.906898    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-certificates\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907024 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.906955    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-installation-pull-secrets\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907024 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.906985    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907024 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.907016    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-image-registry-private-configuration\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907273 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.907076    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgddq\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-kube-api-access-mgddq\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907627 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.907176    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c34e8de-fd01-40a2-81d3-52ece5d86c90-ca-trust-extracted\") pod \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\" (UID: \"4c34e8de-fd01-40a2-81d3-52ece5d86c90\") "
Jul 24 16:48:15.907627 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.907396    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:48:15.907821 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.907473    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:48:15.908677 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.908629    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-trusted-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:15.908677 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.908655    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-certificates\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:15.909964 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.909884    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:48:15.910063 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.910008    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:48:15.910127 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.910059    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-kube-api-access-mgddq" (OuterVolumeSpecName: "kube-api-access-mgddq") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "kube-api-access-mgddq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:48:15.910127 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.910100    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:48:15.910127 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.910113    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:48:15.920973 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:15.916638    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c34e8de-fd01-40a2-81d3-52ece5d86c90-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4c34e8de-fd01-40a2-81d3-52ece5d86c90" (UID: "4c34e8de-fd01-40a2-81d3-52ece5d86c90"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:48:16.009489 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.009452    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-installation-pull-secrets\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:16.009489 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.009486    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-registry-tls\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:16.009489 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.009497    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4c34e8de-fd01-40a2-81d3-52ece5d86c90-image-registry-private-configuration\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:16.009489 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.009508    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgddq\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-kube-api-access-mgddq\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:16.009758 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.009517    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4c34e8de-fd01-40a2-81d3-52ece5d86c90-ca-trust-extracted\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:16.009758 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.009526    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c34e8de-fd01-40a2-81d3-52ece5d86c90-bound-sa-token\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:48:16.580539 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.580503    2569 generic.go:356] "Generic (PLEG): container finished" podID="4c34e8de-fd01-40a2-81d3-52ece5d86c90" containerID="8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7" exitCode=0
Jul 24 16:48:16.580924 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.580551    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq"
Jul 24 16:48:16.580924 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.580589    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq" event={"ID":"4c34e8de-fd01-40a2-81d3-52ece5d86c90","Type":"ContainerDied","Data":"8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7"}
Jul 24 16:48:16.580924 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.580671    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbb7dbb8d-6jskq" event={"ID":"4c34e8de-fd01-40a2-81d3-52ece5d86c90","Type":"ContainerDied","Data":"5dcaf328c5e702bf5ba079b58d86e1e1b3cec6b71788fed5b2736bba4a0b3d42"}
Jul 24 16:48:16.580924 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.580696    2569 scope.go:117] "RemoveContainer" containerID="8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7"
Jul 24 16:48:16.589752 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.589736    2569 scope.go:117] "RemoveContainer" containerID="8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7"
Jul 24 16:48:16.590002 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:48:16.589983    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7\": container with ID starting with 8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7 not found: ID does not exist" containerID="8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7"
Jul 24 16:48:16.590050 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.590011    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7"} err="failed to get container status \"8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7\": rpc error: code = NotFound desc = could not find container \"8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7\": container with ID starting with 8c5ade764c84d736bb14252969b34de01e7e278aa3e4ccea68bffc686eb8dcd7 not found: ID does not exist"
Jul 24 16:48:16.601648 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.601629    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bbb7dbb8d-6jskq"]
Jul 24 16:48:16.605319 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:16.605297    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bbb7dbb8d-6jskq"]
Jul 24 16:48:17.629237 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:17.629202    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c34e8de-fd01-40a2-81d3-52ece5d86c90" path="/var/lib/kubelet/pods/4c34e8de-fd01-40a2-81d3-52ece5d86c90/volumes"
Jul 24 16:48:19.624753 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:19.624719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:48:48.696001 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:48.695971    2569 generic.go:356] "Generic (PLEG): container finished" podID="d473c729-80ec-4439-9407-df3b25da6534" containerID="f6a11eb2587c3e0ce459bdf3bc64e05371355d1dc42eabd91345e24eefdb4605" exitCode=0
Jul 24 16:48:48.696408 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:48.696039    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm" event={"ID":"d473c729-80ec-4439-9407-df3b25da6534","Type":"ContainerDied","Data":"f6a11eb2587c3e0ce459bdf3bc64e05371355d1dc42eabd91345e24eefdb4605"}
Jul 24 16:48:48.696408 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:48.696321    2569 scope.go:117] "RemoveContainer" containerID="f6a11eb2587c3e0ce459bdf3bc64e05371355d1dc42eabd91345e24eefdb4605"
Jul 24 16:48:49.702133 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:48:49.702098    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-mn2vm" event={"ID":"d473c729-80ec-4439-9407-df3b25da6534","Type":"ContainerStarted","Data":"c0e5006c3a97409711b34b0d6584fb3f5bc618e0bc7ac267a9afc4d070853994"}
Jul 24 16:49:05.509422 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:05.509379    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:05.527887 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:05.527854    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:05.775829 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:05.775726    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:23.598212 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.598170    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:49:23.598832 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.598575    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="prometheus" containerID="cri-o://87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5" gracePeriod=600
Jul 24 16:49:23.598832 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.598613    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="config-reloader" containerID="cri-o://13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a" gracePeriod=600
Jul 24 16:49:23.598832 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.598603    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="thanos-sidecar" containerID="cri-o://e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6" gracePeriod=600
Jul 24 16:49:23.598832 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.598622    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402" gracePeriod=600
Jul 24 16:49:23.598832 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.598576    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-web" containerID="cri-o://a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140" gracePeriod=600
Jul 24 16:49:23.598832 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.598622    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy" containerID="cri-o://dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa" gracePeriod=600
Jul 24 16:49:23.830984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.830946    2569 generic.go:356] "Generic (PLEG): container finished" podID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerID="e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402" exitCode=0
Jul 24 16:49:23.830984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.830974    2569 generic.go:356] "Generic (PLEG): container finished" podID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerID="dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa" exitCode=0
Jul 24 16:49:23.830984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.830982    2569 generic.go:356] "Generic (PLEG): container finished" podID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerID="e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6" exitCode=0
Jul 24 16:49:23.830984 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.830989    2569 generic.go:356] "Generic (PLEG): container finished" podID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerID="13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a" exitCode=0
Jul 24 16:49:23.831235 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.830998    2569 generic.go:356] "Generic (PLEG): container finished" podID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerID="87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5" exitCode=0
Jul 24 16:49:23.831235 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.831018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402"}
Jul 24 16:49:23.831235 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.831080    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa"}
Jul 24 16:49:23.831235 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.831108    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6"}
Jul 24 16:49:23.831235 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.831132    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a"}
Jul 24 16:49:23.831235 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:23.831156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5"}
Jul 24 16:49:24.835200 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.835175    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:24.837050 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.837026    2569 generic.go:356] "Generic (PLEG): container finished" podID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerID="a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140" exitCode=0
Jul 24 16:49:24.837145 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.837096    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140"}
Jul 24 16:49:24.837203 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.837158    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7e271f6-f94d-459a-8c9e-75c393e7d4b3","Type":"ContainerDied","Data":"66995050b78513b042c3a58f15bb3878515511c08ee1eda9369ace99670c7b8f"}
Jul 24 16:49:24.837203 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.837194    2569 scope.go:117] "RemoveContainer" containerID="e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402"
Jul 24 16:49:24.844585 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.844568    2569 scope.go:117] "RemoveContainer" containerID="dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa"
Jul 24 16:49:24.851156 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.851139    2569 scope.go:117] "RemoveContainer" containerID="a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140"
Jul 24 16:49:24.857962 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.857944    2569 scope.go:117] "RemoveContainer" containerID="e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6"
Jul 24 16:49:24.864536 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.864513    2569 scope.go:117] "RemoveContainer" containerID="13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a"
Jul 24 16:49:24.871490 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.871467    2569 scope.go:117] "RemoveContainer" containerID="87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5"
Jul 24 16:49:24.879566 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.879535    2569 scope.go:117] "RemoveContainer" containerID="f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3"
Jul 24 16:49:24.887619 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.887592    2569 scope.go:117] "RemoveContainer" containerID="e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402"
Jul 24 16:49:24.887887 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:49:24.887866    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402\": container with ID starting with e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402 not found: ID does not exist" containerID="e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402"
Jul 24 16:49:24.887967 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.887895    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402"} err="failed to get container status \"e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402\": rpc error: code = NotFound desc = could not find container \"e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402\": container with ID starting with e3bd11185130ea9bc5bec1cc33a39a80eef5dfa28345709571222eb14186c402 not found: ID does not exist"
Jul 24 16:49:24.887967 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.887915    2569 scope.go:117] "RemoveContainer" containerID="dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa"
Jul 24 16:49:24.888242 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:49:24.888194    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa\": container with ID starting with dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa not found: ID does not exist" containerID="dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa"
Jul 24 16:49:24.888372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.888243    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa"} err="failed to get container status \"dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa\": rpc error: code = NotFound desc = could not find container \"dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa\": container with ID starting with dbff6d29883b613ce9ba9f3fe19509de3500ef9e00ed41473ca9fc22cfde7faa not found: ID does not exist"
Jul 24 16:49:24.888372 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.888264    2569 scope.go:117] "RemoveContainer" containerID="a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140"
Jul 24 16:49:24.888602 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:49:24.888572    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140\": container with ID starting with a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140 not found: ID does not exist" containerID="a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140"
Jul 24 16:49:24.888702 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.888611    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140"} err="failed to get container status \"a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140\": rpc error: code = NotFound desc = could not find container \"a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140\": container with ID starting with a55697a2b776c1f097d46ad6b949412bb28867e252e17765aa2e7a7cfbf49140 not found: ID does not exist"
Jul 24 16:49:24.888702 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.888632    2569 scope.go:117] "RemoveContainer" containerID="e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6"
Jul 24 16:49:24.888922 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:49:24.888904    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6\": container with ID starting with e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6 not found: ID does not exist" containerID="e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6"
Jul 24 16:49:24.888982 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.888928    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6"} err="failed to get container status \"e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6\": rpc error: code = NotFound desc = could not find container \"e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6\": container with ID starting with e69a4523fcfecb4151185fff6044d75aaead23fb31fb2bd9264291a2069d03b6 not found: ID does not exist"
Jul 24 16:49:24.888982 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.888948    2569 scope.go:117] "RemoveContainer" containerID="13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a"
Jul 24 16:49:24.889161 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:49:24.889143    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a\": container with ID starting with 13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a not found: ID does not exist" containerID="13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a"
Jul 24 16:49:24.889218 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.889164    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a"} err="failed to get container status \"13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a\": rpc error: code = NotFound desc = could not find container \"13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a\": container with ID starting with 13b4058b38efbb98c3ea892cdc7d67eabda4e977022051dc73970eadd652135a not found: ID does not exist"
Jul 24 16:49:24.889218 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.889177    2569 scope.go:117] "RemoveContainer" containerID="87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5"
Jul 24 16:49:24.889385 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:49:24.889369    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5\": container with ID starting with 87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5 not found: ID does not exist" containerID="87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5"
Jul 24 16:49:24.889451 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.889389    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5"} err="failed to get container status \"87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5\": rpc error: code = NotFound desc = could not find container \"87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5\": container with ID starting with 87396c1fd70b8a9af26ae0586dd2fa54c144a9c0ba4ad1168bc7766a6236eaa5 not found: ID does not exist"
Jul 24 16:49:24.889451 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.889401    2569 scope.go:117] "RemoveContainer" containerID="f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3"
Jul 24 16:49:24.889571 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:49:24.889555    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3\": container with ID starting with f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3 not found: ID does not exist" containerID="f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3"
Jul 24 16:49:24.889626 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.889574    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3"} err="failed to get container status \"f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3\": rpc error: code = NotFound desc = could not find container \"f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3\": container with ID starting with f07799f3a0e94bb06bd8f5b6113dc8816d20bb12436d50a0f777e7d874f5c6e3 not found: ID does not exist"
Jul 24 16:49:24.911100 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911027    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911100 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911083    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-tls-assets\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911112    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config-out\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-metrics-client-certs\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911170    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-grpc-tls\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911209    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911236    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-web-config\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911263    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-kube-rbac-proxy\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911287    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-2\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911323    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-db\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911366    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-1\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911410    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911441    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-metrics-client-ca\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911470    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-0\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911534    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-thanos-prometheus-http-client-file\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911564    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-serving-certs-ca-bundle\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911590    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-tls\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-kubelet-serving-ca-bundle\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.911666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.911641    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7qzw\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-kube-api-access-q7qzw\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.912466    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-trusted-ca-bundle\") pod \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\" (UID: \"d7e271f6-f94d-459a-8c9e-75c393e7d4b3\") "
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.912859    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.912324    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.913495    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.913752    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.913786    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.914135    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.914427    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.914509    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.915390    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.915644    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.915990    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-kube-api-access-q7qzw" (OuterVolumeSpecName: "kube-api-access-q7qzw") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "kube-api-access-q7qzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.916027    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config-out" (OuterVolumeSpecName: "config-out") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:49:24.916508 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.916072    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.917258 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.916249    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.917258 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.916271    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.917258 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.917090    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config" (OuterVolumeSpecName: "config") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.917468 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.917443    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.917526 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.917483    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:49:24.917980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.917956    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:24.927340 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:24.927310    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-web-config" (OuterVolumeSpecName: "web-config") pod "d7e271f6-f94d-459a-8c9e-75c393e7d4b3" (UID: "d7e271f6-f94d-459a-8c9e-75c393e7d4b3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:49:25.013140 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013091    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-db\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013140 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013133    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013140 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013145    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013140 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013156    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-metrics-client-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013165    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013174    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013182    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013191    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013199    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013207    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7qzw\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-kube-api-access-q7qzw\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013215    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013223    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013233    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-tls-assets\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013240    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config-out\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013248    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-metrics-client-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013257    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-grpc-tls\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013264    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-config\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013272    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-web-config\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013279    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-secret-kube-rbac-proxy\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.013432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.013288    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7e271f6-f94d-459a-8c9e-75c393e7d4b3-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:49:25.841691 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.841656    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.862249 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.862220    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:49:25.866179 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.866158    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:49:25.898963 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.898891    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:49:25.899174 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899162    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="prometheus"
Jul 24 16:49:25.899214 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899176    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="prometheus"
Jul 24 16:49:25.899214 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899190    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-thanos"
Jul 24 16:49:25.899214 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899199    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-thanos"
Jul 24 16:49:25.899214 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899209    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899215    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899224    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-web"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899230    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-web"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899238    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c34e8de-fd01-40a2-81d3-52ece5d86c90" containerName="registry"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899242    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c34e8de-fd01-40a2-81d3-52ece5d86c90" containerName="registry"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899250    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="config-reloader"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899255    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="config-reloader"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899264    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="init-config-reloader"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899273    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="init-config-reloader"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899284    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="thanos-sidecar"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899289    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="thanos-sidecar"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899340    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-web"
Jul 24 16:49:25.899351 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899348    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="prometheus"
Jul 24 16:49:25.899696 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899361    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c34e8de-fd01-40a2-81d3-52ece5d86c90" containerName="registry"
Jul 24 16:49:25.899696 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899369    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy"
Jul 24 16:49:25.899696 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899375    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="config-reloader"
Jul 24 16:49:25.899696 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899380    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="thanos-sidecar"
Jul 24 16:49:25.899696 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.899385    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" containerName="kube-rbac-proxy-thanos"
Jul 24 16:49:25.904483 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.904467    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.906904 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.906882    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 16:49:25.906904 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.906895    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 16:49:25.907079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.906896    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 16:49:25.907079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.906941    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-48bt9\""
Jul 24 16:49:25.907079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.906944    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 16:49:25.907079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.906962    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 16:49:25.907079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.906886    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 16:49:25.907355 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.907323    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 16:49:25.907605 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.907587    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 16:49:25.907684 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.907588    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 16:49:25.907877 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.907861    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 16:49:25.908255 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.908238    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 16:49:25.908321 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.908248    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 16:49:25.908571 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.908555    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bi021q3t25spa\""
Jul 24 16:49:25.908639 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.908561    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 16:49:25.915240 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.912033    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 16:49:25.916735 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.916716    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 16:49:25.918068 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918044    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918171 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918171 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918171 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918171 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-config\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918225    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a09a5023-7c67-4d22-a66d-06f64ce29c51-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqxqc\" (UniqueName: \"kubernetes.io/projected/a09a5023-7c67-4d22-a66d-06f64ce29c51-kube-api-access-nqxqc\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918338    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-web-config\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918392 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918381    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918658 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918401    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918658 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918431    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918658 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918452    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918658 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918658 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918512    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918658 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918550    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918658 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918631    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918899 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.918899 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.918701    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a09a5023-7c67-4d22-a66d-06f64ce29c51-config-out\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:25.919652 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:25.919631    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:49:26.019241 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019241 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019497 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019497 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019386    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019497 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019429    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019497 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a09a5023-7c67-4d22-a66d-06f64ce29c51-config-out\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019530    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019642    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-config\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a09a5023-7c67-4d22-a66d-06f64ce29c51-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019732    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqxqc\" (UniqueName: \"kubernetes.io/projected/a09a5023-7c67-4d22-a66d-06f64ce29c51-kube-api-access-nqxqc\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019747    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-web-config\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019761    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.019993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.019852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.020418 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.020162    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.020418 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.020160    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.020418 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.020286    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.020418 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.020301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.021913 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.020979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.021913 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.021521    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.022577 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.022388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.022897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.022875    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.022997 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.022972    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-web-config\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.023064 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.023005    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.023064 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.023030    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.023177 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.023105    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.023586 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.023565    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a09a5023-7c67-4d22-a66d-06f64ce29c51-config-out\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.023817 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.023794    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.024792 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.024771    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-config\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.025297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.025278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a09a5023-7c67-4d22-a66d-06f64ce29c51-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.025397 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.025314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a09a5023-7c67-4d22-a66d-06f64ce29c51-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.025397 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.025356    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.025851 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.025832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a09a5023-7c67-4d22-a66d-06f64ce29c51-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.028003 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.027982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqxqc\" (UniqueName: \"kubernetes.io/projected/a09a5023-7c67-4d22-a66d-06f64ce29c51-kube-api-access-nqxqc\") pod \"prometheus-k8s-0\" (UID: \"a09a5023-7c67-4d22-a66d-06f64ce29c51\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.218951 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.218866    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:49:26.354621 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.354581    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 16:49:26.358243 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:49:26.358220    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09a5023_7c67_4d22_a66d_06f64ce29c51.slice/crio-f29dc1582b8a6fd5e7377907136d3da4b604f94c7067a3a6566a6ec554a4b6d0 WatchSource:0}: Error finding container f29dc1582b8a6fd5e7377907136d3da4b604f94c7067a3a6566a6ec554a4b6d0: Status 404 returned error can't find the container with id f29dc1582b8a6fd5e7377907136d3da4b604f94c7067a3a6566a6ec554a4b6d0
Jul 24 16:49:26.846682 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.846648    2569 generic.go:356] "Generic (PLEG): container finished" podID="a09a5023-7c67-4d22-a66d-06f64ce29c51" containerID="456b2644f0d30f037613251dc4d95c05e67fb997df92836707b9cd56a84c3fd1" exitCode=0
Jul 24 16:49:26.847052 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.846711    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerDied","Data":"456b2644f0d30f037613251dc4d95c05e67fb997df92836707b9cd56a84c3fd1"}
Jul 24 16:49:26.847052 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:26.846762    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerStarted","Data":"f29dc1582b8a6fd5e7377907136d3da4b604f94c7067a3a6566a6ec554a4b6d0"}
Jul 24 16:49:27.629194 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.629164    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e271f6-f94d-459a-8c9e-75c393e7d4b3" path="/var/lib/kubelet/pods/d7e271f6-f94d-459a-8c9e-75c393e7d4b3/volumes"
Jul 24 16:49:27.634003 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.633978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:49:27.639352 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.637599    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/12520acc-f016-4ab9-bc6e-3efd878c185b-metrics-certs\") pod \"network-metrics-daemon-5mppg\" (UID: \"12520acc-f016-4ab9-bc6e-3efd878c185b\") " pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:49:27.727527 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.727490    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6gtf2\""
Jul 24 16:49:27.735688 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.735667    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5mppg"
Jul 24 16:49:27.854744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.854714    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerStarted","Data":"15975b274544474f1d98290b1481154c51bdbb1b5b1f4642e7e6c3c60ee77841"}
Jul 24 16:49:27.855079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.854803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerStarted","Data":"a8b2824ef5ff2d9f670523a808d200e8569ec5c39f6491a65c62e42c04a5afb2"}
Jul 24 16:49:27.855079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.854832    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerStarted","Data":"6701254b385de926d7ae5c8eccdcdc22f194db8e6244623813e37824c157bc62"}
Jul 24 16:49:27.855079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.854856    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerStarted","Data":"ace9737e26a3c42c0459c4791de65eabc7269796d966adb843aaa9c2c267dd7b"}
Jul 24 16:49:27.855079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.854880    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerStarted","Data":"322a4b029c46f99049dbd555e21d4a68f10c2d0106978ae2cb92b555fce0aafa"}
Jul 24 16:49:27.855079 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.854903    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a09a5023-7c67-4d22-a66d-06f64ce29c51","Type":"ContainerStarted","Data":"0add6f86d30ec71a76073f1866c7ed33d068b93f2fa01d2c030d1777ccfc1f95"}
Jul 24 16:49:27.860817 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.860789    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5mppg"]
Jul 24 16:49:27.865701 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:49:27.865678    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12520acc_f016_4ab9_bc6e_3efd878c185b.slice/crio-4638965f24030b6034d6fdf4d1d42a46e649e5d1059a7efd9bb5fbb1e25fea0e WatchSource:0}: Error finding container 4638965f24030b6034d6fdf4d1d42a46e649e5d1059a7efd9bb5fbb1e25fea0e: Status 404 returned error can't find the container with id 4638965f24030b6034d6fdf4d1d42a46e649e5d1059a7efd9bb5fbb1e25fea0e
Jul 24 16:49:27.867888 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.867871    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:49:27.883231 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:27.883104    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.8645344809999997 podStartE2EDuration="2.883084297s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.549816ms" isStatefulPod=false podCreationTimestamp="2026-07-24 16:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:49:27.881459074 +0000 UTC m=+374.641268074" watchObservedRunningTime="2026-07-24 16:49:27.883084297 +0000 UTC m=+374.642893298"
Jul 24 16:49:28.860376 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:28.860344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5mppg" event={"ID":"12520acc-f016-4ab9-bc6e-3efd878c185b","Type":"ContainerStarted","Data":"75a9db0622a6b9e22bf12638d3afc86b37cf68f96768c9e6d988c2c913bb4ac2"}
Jul 24 16:49:28.860754 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:28.860405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5mppg" event={"ID":"12520acc-f016-4ab9-bc6e-3efd878c185b","Type":"ContainerStarted","Data":"d4dc1bc67a1495dad73fe90099cef890a0592d3af381919ad01fcba688155786"}
Jul 24 16:49:28.860754 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:28.860422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5mppg" event={"ID":"12520acc-f016-4ab9-bc6e-3efd878c185b","Type":"ContainerStarted","Data":"4638965f24030b6034d6fdf4d1d42a46e649e5d1059a7efd9bb5fbb1e25fea0e"}
Jul 24 16:49:28.876608 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:28.876563    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5mppg" podStartSLOduration=375.056415827 podStartE2EDuration="6m15.876547887s" totalImagesPullingTime="820.13206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:49:28.87628458 +0000 UTC m=+375.636093578" watchObservedRunningTime="2026-07-24 16:49:28.876547887 +0000 UTC m=+375.636356886"
Jul 24 16:49:31.219524 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:49:31.219473    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:50:26.220010 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:50:26.219970    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:50:26.235679 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:50:26.235650    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:50:27.081864 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:50:27.081838    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 16:52:50.779266 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.779228    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"]
Jul 24 16:52:50.782338 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.782313    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:50.785968 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.785865    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 16:52:50.785968 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.785882    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r7w7p\""
Jul 24 16:52:50.785968 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.785902    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 16:52:50.785968 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.785895    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 16:52:50.786232 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.786017    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 16:52:50.786232 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.786147    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 16:52:50.793287 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.793263    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"]
Jul 24 16:52:50.920373 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.920316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d89369-25e6-4def-a8e6-a2b24d7764f5-cert\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:50.920560 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.920448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90d89369-25e6-4def-a8e6-a2b24d7764f5-manager-config\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:50.920560 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.920472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90d89369-25e6-4def-a8e6-a2b24d7764f5-metrics-cert\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:50.920560 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:50.920490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b822k\" (UniqueName: \"kubernetes.io/projected/90d89369-25e6-4def-a8e6-a2b24d7764f5-kube-api-access-b822k\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.021484 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.021448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90d89369-25e6-4def-a8e6-a2b24d7764f5-manager-config\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.021484 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.021481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90d89369-25e6-4def-a8e6-a2b24d7764f5-metrics-cert\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.021782 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.021504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b822k\" (UniqueName: \"kubernetes.io/projected/90d89369-25e6-4def-a8e6-a2b24d7764f5-kube-api-access-b822k\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.021782 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.021552    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d89369-25e6-4def-a8e6-a2b24d7764f5-cert\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.022216 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.022190    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90d89369-25e6-4def-a8e6-a2b24d7764f5-manager-config\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.024037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.024016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90d89369-25e6-4def-a8e6-a2b24d7764f5-metrics-cert\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.024130 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.024069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90d89369-25e6-4def-a8e6-a2b24d7764f5-cert\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.030062 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.030013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b822k\" (UniqueName: \"kubernetes.io/projected/90d89369-25e6-4def-a8e6-a2b24d7764f5-kube-api-access-b822k\") pod \"lws-controller-manager-675dd78886-stwrf\" (UID: \"90d89369-25e6-4def-a8e6-a2b24d7764f5\") " pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.092216 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.092175    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:51.241539 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.241410    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"]
Jul 24 16:52:51.244115 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:52:51.244089    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d89369_25e6_4def_a8e6_a2b24d7764f5.slice/crio-6cada616752c6478665db41ebdcf91f5776144f74384b1e55ed3df4afb292f39 WatchSource:0}: Error finding container 6cada616752c6478665db41ebdcf91f5776144f74384b1e55ed3df4afb292f39: Status 404 returned error can't find the container with id 6cada616752c6478665db41ebdcf91f5776144f74384b1e55ed3df4afb292f39
Jul 24 16:52:51.590007 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:51.589975    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf" event={"ID":"90d89369-25e6-4def-a8e6-a2b24d7764f5","Type":"ContainerStarted","Data":"6cada616752c6478665db41ebdcf91f5776144f74384b1e55ed3df4afb292f39"}
Jul 24 16:52:52.932661 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.932629    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"]
Jul 24 16:52:52.941206 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.941183    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:52.943699 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.943670    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pvp4m\""
Jul 24 16:52:52.943973 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.943957    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 16:52:52.944367 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.944201    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 16:52:52.944367 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.944207    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 16:52:52.944367 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.944207    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 16:52:52.952743 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:52.952708    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"]
Jul 24 16:52:53.039311 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.039273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpsxr\" (UniqueName: \"kubernetes.io/projected/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-kube-api-access-kpsxr\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.039517 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.039357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.039517 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.039392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-manifests\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.039517 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.039475    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-tmp\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.039517 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.039506    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.140594 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.140555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.140779 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.140603    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-manifests\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.140779 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.140666    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-tmp\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.140779 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.140701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.140779 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.140762    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpsxr\" (UniqueName: \"kubernetes.io/projected/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-kube-api-access-kpsxr\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.141108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.141085    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-manifests\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.143425 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.143395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-tmp\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.143889 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.143864    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.144003 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.143912    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-webhook-cert\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.149664 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.149635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpsxr\" (UniqueName: \"kubernetes.io/projected/2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7-kube-api-access-kpsxr\") pod \"opendatahub-operator-controller-manager-769b58b787-gq457\" (UID: \"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7\") " pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.253946 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.253812    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:53.700891 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:53.700865    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"]
Jul 24 16:52:53.703666 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:52:53.703642    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c77c503_ed9d_4f0d_9aaf_51ae063fd9d7.slice/crio-4e679fb0aee1d6cd28a5d40247e07e968783e126db3f9afb11edb3192964fc4f WatchSource:0}: Error finding container 4e679fb0aee1d6cd28a5d40247e07e968783e126db3f9afb11edb3192964fc4f: Status 404 returned error can't find the container with id 4e679fb0aee1d6cd28a5d40247e07e968783e126db3f9afb11edb3192964fc4f
Jul 24 16:52:54.606579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:54.606534    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf" event={"ID":"90d89369-25e6-4def-a8e6-a2b24d7764f5","Type":"ContainerStarted","Data":"c00f93ced087519d540cbf6cdbeb068abe53ff330fd963ca77178cc80920fb39"}
Jul 24 16:52:54.607025 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:54.606602    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:52:54.608225 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:54.608191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457" event={"ID":"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7","Type":"ContainerStarted","Data":"4e679fb0aee1d6cd28a5d40247e07e968783e126db3f9afb11edb3192964fc4f"}
Jul 24 16:52:54.640293 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:54.640241    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf" podStartSLOduration=2.257970446 podStartE2EDuration="4.640217656s" totalImagesPullingTime="2.38224721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:52:54.63761247 +0000 UTC m=+581.397421473" watchObservedRunningTime="2026-07-24 16:52:54.640217656 +0000 UTC m=+581.400026657"
Jul 24 16:52:57.621208 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:57.621169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457" event={"ID":"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7","Type":"ContainerStarted","Data":"73a73e5b78ce7fe9a879dd3c8a59d51eed0b892ef5c39e415262e05d77c54666"}
Jul 24 16:52:58.626798 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:58.626763    2569 generic.go:356] "Generic (PLEG): container finished" podID="2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7" containerID="73a73e5b78ce7fe9a879dd3c8a59d51eed0b892ef5c39e415262e05d77c54666" exitCode=0
Jul 24 16:52:58.627170 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:58.626855    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457" event={"ID":"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7","Type":"ContainerDied","Data":"73a73e5b78ce7fe9a879dd3c8a59d51eed0b892ef5c39e415262e05d77c54666"}
Jul 24 16:52:59.632283 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:59.632253    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457" event={"ID":"2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7","Type":"ContainerStarted","Data":"6c0d4cb60fd499eebed86c84f0a13358ddf6c99fc0f0f37eacb9f3bcc678709b"}
Jul 24 16:52:59.632664 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:59.632399    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:52:59.654837 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:52:59.654790    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457" podStartSLOduration=3.283268281 podStartE2EDuration="7.654776141s" totalImagesPullingTime="3.279677398s" totalInitContainerRuntime="1.091830462s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:52:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:52:59.653040945 +0000 UTC m=+586.412849943" watchObservedRunningTime="2026-07-24 16:52:59.654776141 +0000 UTC m=+586.414585140"
Jul 24 16:53:05.614045 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:05.614012    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-675dd78886-stwrf"
Jul 24 16:53:10.638708 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:10.638674    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-769b58b787-gq457"
Jul 24 16:53:13.170169 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.170131    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"]
Jul 24 16:53:13.270728 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.270689    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"]
Jul 24 16:53:13.270887 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.270784    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.273413 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.273385    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 16:53:13.273711 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.273695    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 16:53:13.273891 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.273873    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f9z7v\""
Jul 24 16:53:13.274147 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.274132    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 16:53:13.274191 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.274159    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 16:53:13.316746 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.316706    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-tmp\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.316897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.316758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-tls-certs\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.316897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.316813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8k95\" (UniqueName: \"kubernetes.io/projected/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-kube-api-access-r8k95\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.417649 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.417615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-tmp\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.417649 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.417653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-tls-certs\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.417891 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.417681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8k95\" (UniqueName: \"kubernetes.io/projected/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-kube-api-access-r8k95\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.419914 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.419891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-tmp\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.420125 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.420107    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-tls-certs\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.442908 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.442843    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8k95\" (UniqueName: \"kubernetes.io/projected/7244aa23-35f3-4163-b83f-fa7fdbd6e97f-kube-api-access-r8k95\") pod \"kube-auth-proxy-6cb5df4894-ctb7g\" (UID: \"7244aa23-35f3-4163-b83f-fa7fdbd6e97f\") " pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.582360 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.582320    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f9z7v\""
Jul 24 16:53:13.590969 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.590947    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"
Jul 24 16:53:13.788643 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:13.788612    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g"]
Jul 24 16:53:13.792455 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:53:13.792428    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7244aa23_35f3_4163_b83f_fa7fdbd6e97f.slice/crio-087fe88085417d505683d2675c68c3cb5174bef8708d8eb63d1d9281d4a4598e WatchSource:0}: Error finding container 087fe88085417d505683d2675c68c3cb5174bef8708d8eb63d1d9281d4a4598e: Status 404 returned error can't find the container with id 087fe88085417d505683d2675c68c3cb5174bef8708d8eb63d1d9281d4a4598e
Jul 24 16:53:14.687415 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:14.687381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g" event={"ID":"7244aa23-35f3-4163-b83f-fa7fdbd6e97f","Type":"ContainerStarted","Data":"087fe88085417d505683d2675c68c3cb5174bef8708d8eb63d1d9281d4a4598e"}
Jul 24 16:53:16.170104 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:16.170086    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 16:53:16.697445 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:16.697409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g" event={"ID":"7244aa23-35f3-4163-b83f-fa7fdbd6e97f","Type":"ContainerStarted","Data":"c3b0ea857a94190d4d0b63fa8ad8181d9df7405fdad25c0cfb8083a01a6fcc86"}
Jul 24 16:53:16.713583 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:53:16.713533    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6cb5df4894-ctb7g" podStartSLOduration=1.340079792 podStartE2EDuration="3.713517675s" totalImagesPullingTime="2.373437883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:53:16.712463954 +0000 UTC m=+603.472272945" watchObservedRunningTime="2026-07-24 16:53:16.713517675 +0000 UTC m=+603.473326714"
Jul 24 16:54:57.993261 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:57.993220    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"]
Jul 24 16:54:57.996791 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:57.996765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:57.999242 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:57.999217    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 16:54:57.999419 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:57.999400    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 16:54:57.999867 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:57.999845    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f9j46\""
Jul 24 16:54:58.013318 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.013295    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"]
Jul 24 16:54:58.087182 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.087147    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:58.087371 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.087206    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l5pc\" (UniqueName: \"kubernetes.io/projected/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-kube-api-access-8l5pc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:58.188426 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.188394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:58.188614 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.188438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l5pc\" (UniqueName: \"kubernetes.io/projected/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-kube-api-access-8l5pc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:58.188854 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.188832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:58.199927 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.199898    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l5pc\" (UniqueName: \"kubernetes.io/projected/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-kube-api-access-8l5pc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:58.307208 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.307126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:54:58.440486 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.440459    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"]
Jul 24 16:54:58.443229 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:54:58.443196    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c5dc7f_53a3_4bcb_81d2_fd72189dd008.slice/crio-e9219508b70e795868cfc6e0623841d0fbe8a486d801042a6151a0ff177bce73 WatchSource:0}: Error finding container e9219508b70e795868cfc6e0623841d0fbe8a486d801042a6151a0ff177bce73: Status 404 returned error can't find the container with id e9219508b70e795868cfc6e0623841d0fbe8a486d801042a6151a0ff177bce73
Jul 24 16:54:58.446290 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:58.446273    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:54:59.123080 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:54:59.123043    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56" event={"ID":"34c5dc7f-53a3-4bcb-81d2-fd72189dd008","Type":"ContainerStarted","Data":"e9219508b70e795868cfc6e0623841d0fbe8a486d801042a6151a0ff177bce73"}
Jul 24 16:55:04.148113 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:04.148017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56" event={"ID":"34c5dc7f-53a3-4bcb-81d2-fd72189dd008","Type":"ContainerStarted","Data":"66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36"}
Jul 24 16:55:04.148607 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:04.148167    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:04.186849 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:04.186784    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56" podStartSLOduration=1.736144804 podStartE2EDuration="7.18676753s" totalImagesPullingTime="5.450622726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:55:04.184851128 +0000 UTC m=+710.944660129" watchObservedRunningTime="2026-07-24 16:55:04.18676753 +0000 UTC m=+710.946576560"
Jul 24 16:55:15.154899 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:15.154822    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:16.879021 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.878986    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"]
Jul 24 16:55:16.879507 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.879160    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" containerName="manager" containerID="cri-o://66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36" gracePeriod=2
Jul 24 16:55:16.890309 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.890269    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"]
Jul 24 16:55:16.899390 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:55:16.899351    2569 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-28b56\": pods \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:16.905200 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.905175    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"]
Jul 24 16:55:16.905527 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.905514    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" containerName="manager"
Jul 24 16:55:16.905577 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.905528    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" containerName="manager"
Jul 24 16:55:16.905612 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.905579    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" containerName="manager"
Jul 24 16:55:16.908466 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.908445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:16.922652 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:16.922622    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"]
Jul 24 16:55:16.935416 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:55:16.935368    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" is forbidden: User \"system:node:ip-10-0-135-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-204.ec2.internal' and this object" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:17.056961 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.056920    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35f90af5-01a4-4f79-915d-5373deb4e199-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qcxcm\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:17.057163 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.057134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjrgq\" (UniqueName: \"kubernetes.io/projected/35f90af5-01a4-4f79-915d-5373deb4e199-kube-api-access-tjrgq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qcxcm\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:17.114007 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.113967    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:17.116116 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:55:17.116084    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" is forbidden: User \"system:node:ip-10-0-135-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-204.ec2.internal' and this object" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:17.158666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.158592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjrgq\" (UniqueName: \"kubernetes.io/projected/35f90af5-01a4-4f79-915d-5373deb4e199-kube-api-access-tjrgq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qcxcm\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:17.158666 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.158638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35f90af5-01a4-4f79-915d-5373deb4e199-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qcxcm\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:17.158966 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.158950    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35f90af5-01a4-4f79-915d-5373deb4e199-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qcxcm\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:17.166930 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.166905    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjrgq\" (UniqueName: \"kubernetes.io/projected/35f90af5-01a4-4f79-915d-5373deb4e199-kube-api-access-tjrgq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qcxcm\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:17.204791 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.204760    2569 generic.go:356] "Generic (PLEG): container finished" podID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" containerID="66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36" exitCode=0
Jul 24 16:55:17.204968 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.204800    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:17.204968 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.204859    2569 scope.go:117] "RemoveContainer" containerID="66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36"
Jul 24 16:55:17.207094 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:55:17.207066    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" is forbidden: User \"system:node:ip-10-0-135-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-204.ec2.internal' and this object" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:17.213453 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.213434    2569 scope.go:117] "RemoveContainer" containerID="66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36"
Jul 24 16:55:17.213775 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:55:17.213755    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36\": container with ID starting with 66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36 not found: ID does not exist" containerID="66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36"
Jul 24 16:55:17.213828 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.213785    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36"} err="failed to get container status \"66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36\": rpc error: code = NotFound desc = could not find container \"66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36\": container with ID starting with 66fe25a7b254c2e423eb9b16666a86e2a28da85a126d4b1d9f304f3a3422ab36 not found: ID does not exist"
Jul 24 16:55:17.259830 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.259793    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-extensions-socket-volume\") pod \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") "
Jul 24 16:55:17.259993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.259886    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l5pc\" (UniqueName: \"kubernetes.io/projected/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-kube-api-access-8l5pc\") pod \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\" (UID: \"34c5dc7f-53a3-4bcb-81d2-fd72189dd008\") "
Jul 24 16:55:17.260215 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.260195    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34c5dc7f-53a3-4bcb-81d2-fd72189dd008" (UID: "34c5dc7f-53a3-4bcb-81d2-fd72189dd008"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:55:17.261921 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.261901    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-kube-api-access-8l5pc" (OuterVolumeSpecName: "kube-api-access-8l5pc") pod "34c5dc7f-53a3-4bcb-81d2-fd72189dd008" (UID: "34c5dc7f-53a3-4bcb-81d2-fd72189dd008"). InnerVolumeSpecName "kube-api-access-8l5pc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:55:17.270098 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.270072    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:17.368036 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.361271    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-extensions-socket-volume\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:55:17.368036 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.361313    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l5pc\" (UniqueName: \"kubernetes.io/projected/34c5dc7f-53a3-4bcb-81d2-fd72189dd008-kube-api-access-8l5pc\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:55:17.419886 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.419806    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"]
Jul 24 16:55:17.423850 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:55:17.423825    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f90af5_01a4_4f79_915d_5373deb4e199.slice/crio-e919ece35bd62df4ace14d73b85db149cb7f8b1393323041c6f1fbf2d43c3ad1 WatchSource:0}: Error finding container e919ece35bd62df4ace14d73b85db149cb7f8b1393323041c6f1fbf2d43c3ad1: Status 404 returned error can't find the container with id e919ece35bd62df4ace14d73b85db149cb7f8b1393323041c6f1fbf2d43c3ad1
Jul 24 16:55:17.514802 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:55:17.514770    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-28b56\" is forbidden: User \"system:node:ip-10-0-135-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-204.ec2.internal' and this object" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-28b56"
Jul 24 16:55:17.629591 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:17.629555    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c5dc7f-53a3-4bcb-81d2-fd72189dd008" path="/var/lib/kubelet/pods/34c5dc7f-53a3-4bcb-81d2-fd72189dd008/volumes"
Jul 24 16:55:18.210553 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:18.210504    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm" event={"ID":"35f90af5-01a4-4f79-915d-5373deb4e199","Type":"ContainerStarted","Data":"0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559"}
Jul 24 16:55:18.211006 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:18.210585    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:18.211006 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:18.210612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm" event={"ID":"35f90af5-01a4-4f79-915d-5373deb4e199","Type":"ContainerStarted","Data":"e919ece35bd62df4ace14d73b85db149cb7f8b1393323041c6f1fbf2d43c3ad1"}
Jul 24 16:55:18.232931 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:18.232879    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm" podStartSLOduration=2.232857778 podStartE2EDuration="2.232857778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:55:18.230646783 +0000 UTC m=+724.990455781" watchObservedRunningTime="2026-07-24 16:55:18.232857778 +0000 UTC m=+724.992666777"
Jul 24 16:55:29.218419 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:29.218390    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:32.876027 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:32.875991    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"]
Jul 24 16:55:32.876470 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:32.876213    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm" podUID="35f90af5-01a4-4f79-915d-5373deb4e199" containerName="manager" containerID="cri-o://0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559" gracePeriod=10
Jul 24 16:55:33.071144 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.071114    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"]
Jul 24 16:55:33.074386 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.074360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.086557 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.086529    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"]
Jul 24 16:55:33.089112 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.089084    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzc82\" (UniqueName: \"kubernetes.io/projected/a5649270-b49e-4015-bed3-3f7f4ac4196d-kube-api-access-jzc82\") pod \"kuadrant-operator-controller-manager-86878c769f-ptxs4\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.089229 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.089188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5649270-b49e-4015-bed3-3f7f4ac4196d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ptxs4\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.125622 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.125596    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:33.190492 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.190416    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjrgq\" (UniqueName: \"kubernetes.io/projected/35f90af5-01a4-4f79-915d-5373deb4e199-kube-api-access-tjrgq\") pod \"35f90af5-01a4-4f79-915d-5373deb4e199\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") "
Jul 24 16:55:33.190638 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.190495    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35f90af5-01a4-4f79-915d-5373deb4e199-extensions-socket-volume\") pod \"35f90af5-01a4-4f79-915d-5373deb4e199\" (UID: \"35f90af5-01a4-4f79-915d-5373deb4e199\") "
Jul 24 16:55:33.190638 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.190579    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzc82\" (UniqueName: \"kubernetes.io/projected/a5649270-b49e-4015-bed3-3f7f4ac4196d-kube-api-access-jzc82\") pod \"kuadrant-operator-controller-manager-86878c769f-ptxs4\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.190638 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.190635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5649270-b49e-4015-bed3-3f7f4ac4196d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ptxs4\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.190870 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.190847    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35f90af5-01a4-4f79-915d-5373deb4e199-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "35f90af5-01a4-4f79-915d-5373deb4e199" (UID: "35f90af5-01a4-4f79-915d-5373deb4e199"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:55:33.190980 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.190961    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5649270-b49e-4015-bed3-3f7f4ac4196d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ptxs4\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.192608 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.192576    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35f90af5-01a4-4f79-915d-5373deb4e199-kube-api-access-tjrgq" (OuterVolumeSpecName: "kube-api-access-tjrgq") pod "35f90af5-01a4-4f79-915d-5373deb4e199" (UID: "35f90af5-01a4-4f79-915d-5373deb4e199"). InnerVolumeSpecName "kube-api-access-tjrgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:55:33.198857 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.198832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzc82\" (UniqueName: \"kubernetes.io/projected/a5649270-b49e-4015-bed3-3f7f4ac4196d-kube-api-access-jzc82\") pod \"kuadrant-operator-controller-manager-86878c769f-ptxs4\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.278192 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.278158    2569 generic.go:356] "Generic (PLEG): container finished" podID="35f90af5-01a4-4f79-915d-5373deb4e199" containerID="0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559" exitCode=0
Jul 24 16:55:33.278394 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.278203    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"
Jul 24 16:55:33.278394 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.278244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm" event={"ID":"35f90af5-01a4-4f79-915d-5373deb4e199","Type":"ContainerDied","Data":"0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559"}
Jul 24 16:55:33.278394 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.278292    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm" event={"ID":"35f90af5-01a4-4f79-915d-5373deb4e199","Type":"ContainerDied","Data":"e919ece35bd62df4ace14d73b85db149cb7f8b1393323041c6f1fbf2d43c3ad1"}
Jul 24 16:55:33.278394 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.278315    2569 scope.go:117] "RemoveContainer" containerID="0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559"
Jul 24 16:55:33.287461 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.287417    2569 scope.go:117] "RemoveContainer" containerID="0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559"
Jul 24 16:55:33.287685 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:55:33.287666    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559\": container with ID starting with 0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559 not found: ID does not exist" containerID="0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559"
Jul 24 16:55:33.287762 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.287691    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559"} err="failed to get container status \"0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559\": rpc error: code = NotFound desc = could not find container \"0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559\": container with ID starting with 0e37e40216b26b976071c7090aae0f5df926f4853c501aa693adcc67be37c559 not found: ID does not exist"
Jul 24 16:55:33.291229 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.291210    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35f90af5-01a4-4f79-915d-5373deb4e199-extensions-socket-volume\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:55:33.291305 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.291230    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjrgq\" (UniqueName: \"kubernetes.io/projected/35f90af5-01a4-4f79-915d-5373deb4e199-kube-api-access-tjrgq\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:55:33.300435 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.300414    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"]
Jul 24 16:55:33.313371 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.313342    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qcxcm"]
Jul 24 16:55:33.384688 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.384658    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:33.521649 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.521616    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"]
Jul 24 16:55:33.525419 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:55:33.525395    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5649270_b49e_4015_bed3_3f7f4ac4196d.slice/crio-99ab7b3c9dcf6a2768112102fbbcc41d79353a908fd875ac1e9114c229f3fb37 WatchSource:0}: Error finding container 99ab7b3c9dcf6a2768112102fbbcc41d79353a908fd875ac1e9114c229f3fb37: Status 404 returned error can't find the container with id 99ab7b3c9dcf6a2768112102fbbcc41d79353a908fd875ac1e9114c229f3fb37
Jul 24 16:55:33.630362 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:33.630312    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35f90af5-01a4-4f79-915d-5373deb4e199" path="/var/lib/kubelet/pods/35f90af5-01a4-4f79-915d-5373deb4e199/volumes"
Jul 24 16:55:34.284631 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:34.284595    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4" event={"ID":"a5649270-b49e-4015-bed3-3f7f4ac4196d","Type":"ContainerStarted","Data":"ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98"}
Jul 24 16:55:34.285014 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:34.284653    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4" event={"ID":"a5649270-b49e-4015-bed3-3f7f4ac4196d","Type":"ContainerStarted","Data":"99ab7b3c9dcf6a2768112102fbbcc41d79353a908fd875ac1e9114c229f3fb37"}
Jul 24 16:55:34.285014 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:34.284723    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:55:34.304183 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:34.304137    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4" podStartSLOduration=1.304122567 podStartE2EDuration="1.304122567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:55:34.302296291 +0000 UTC m=+741.062105303" watchObservedRunningTime="2026-07-24 16:55:34.304122567 +0000 UTC m=+741.063931565"
Jul 24 16:55:45.292694 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:55:45.292664    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 16:56:06.480383 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.480344    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-w44sl"]
Jul 24 16:56:06.480910 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.480806    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35f90af5-01a4-4f79-915d-5373deb4e199" containerName="manager"
Jul 24 16:56:06.480910 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.480825    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="35f90af5-01a4-4f79-915d-5373deb4e199" containerName="manager"
Jul 24 16:56:06.481032 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.480944    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="35f90af5-01a4-4f79-915d-5373deb4e199" containerName="manager"
Jul 24 16:56:06.484097 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.484078    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w44sl"
Jul 24 16:56:06.486301 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.486274    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-crqq6\""
Jul 24 16:56:06.489633 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.489607    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-w44sl"]
Jul 24 16:56:06.586870 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.586839    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx5cm\" (UniqueName: \"kubernetes.io/projected/c65c2a43-435b-4da9-8d1a-9d2a550449ab-kube-api-access-fx5cm\") pod \"authorino-58cd955675-w44sl\" (UID: \"c65c2a43-435b-4da9-8d1a-9d2a550449ab\") " pod="kuadrant-system/authorino-58cd955675-w44sl"
Jul 24 16:56:06.687505 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.687473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx5cm\" (UniqueName: \"kubernetes.io/projected/c65c2a43-435b-4da9-8d1a-9d2a550449ab-kube-api-access-fx5cm\") pod \"authorino-58cd955675-w44sl\" (UID: \"c65c2a43-435b-4da9-8d1a-9d2a550449ab\") " pod="kuadrant-system/authorino-58cd955675-w44sl"
Jul 24 16:56:06.695737 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.695707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx5cm\" (UniqueName: \"kubernetes.io/projected/c65c2a43-435b-4da9-8d1a-9d2a550449ab-kube-api-access-fx5cm\") pod \"authorino-58cd955675-w44sl\" (UID: \"c65c2a43-435b-4da9-8d1a-9d2a550449ab\") " pod="kuadrant-system/authorino-58cd955675-w44sl"
Jul 24 16:56:06.733579 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.733504    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-drbcn"]
Jul 24 16:56:06.736790 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.736767    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-drbcn"
Jul 24 16:56:06.741668 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.741640    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-drbcn"]
Jul 24 16:56:06.808744 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.808714    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w44sl"
Jul 24 16:56:06.889074 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.889045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs5hj\" (UniqueName: \"kubernetes.io/projected/eacfe1eb-901f-4e8e-971a-8621f9a23525-kube-api-access-fs5hj\") pod \"authorino-75d556bd77-drbcn\" (UID: \"eacfe1eb-901f-4e8e-971a-8621f9a23525\") " pod="kuadrant-system/authorino-75d556bd77-drbcn"
Jul 24 16:56:06.939196 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.939139    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-w44sl"]
Jul 24 16:56:06.941479 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:56:06.941440    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc65c2a43_435b_4da9_8d1a_9d2a550449ab.slice/crio-fb5e77e4df83a619f0c32e25fb8356a35a493627d5421af0d7f32a529552dff9 WatchSource:0}: Error finding container fb5e77e4df83a619f0c32e25fb8356a35a493627d5421af0d7f32a529552dff9: Status 404 returned error can't find the container with id fb5e77e4df83a619f0c32e25fb8356a35a493627d5421af0d7f32a529552dff9
Jul 24 16:56:06.990397 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.990283    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs5hj\" (UniqueName: \"kubernetes.io/projected/eacfe1eb-901f-4e8e-971a-8621f9a23525-kube-api-access-fs5hj\") pod \"authorino-75d556bd77-drbcn\" (UID: \"eacfe1eb-901f-4e8e-971a-8621f9a23525\") " pod="kuadrant-system/authorino-75d556bd77-drbcn"
Jul 24 16:56:06.999198 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:06.999164    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs5hj\" (UniqueName: \"kubernetes.io/projected/eacfe1eb-901f-4e8e-971a-8621f9a23525-kube-api-access-fs5hj\") pod \"authorino-75d556bd77-drbcn\" (UID: \"eacfe1eb-901f-4e8e-971a-8621f9a23525\") " pod="kuadrant-system/authorino-75d556bd77-drbcn"
Jul 24 16:56:07.047266 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:07.047226    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-drbcn"
Jul 24 16:56:07.179153 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:07.179132    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-drbcn"]
Jul 24 16:56:07.181731 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:56:07.181694    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeacfe1eb_901f_4e8e_971a_8621f9a23525.slice/crio-7b2a631306f15ffd1ba215bfab7bc024a6a730622f8ecad5ed345ea919da89db WatchSource:0}: Error finding container 7b2a631306f15ffd1ba215bfab7bc024a6a730622f8ecad5ed345ea919da89db: Status 404 returned error can't find the container with id 7b2a631306f15ffd1ba215bfab7bc024a6a730622f8ecad5ed345ea919da89db
Jul 24 16:56:07.433871 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:07.433837    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-drbcn" event={"ID":"eacfe1eb-901f-4e8e-971a-8621f9a23525","Type":"ContainerStarted","Data":"7b2a631306f15ffd1ba215bfab7bc024a6a730622f8ecad5ed345ea919da89db"}
Jul 24 16:56:07.434916 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:07.434887    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w44sl" event={"ID":"c65c2a43-435b-4da9-8d1a-9d2a550449ab","Type":"ContainerStarted","Data":"fb5e77e4df83a619f0c32e25fb8356a35a493627d5421af0d7f32a529552dff9"}
Jul 24 16:56:11.460646 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:11.460569    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-drbcn" event={"ID":"eacfe1eb-901f-4e8e-971a-8621f9a23525","Type":"ContainerStarted","Data":"7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad"}
Jul 24 16:56:11.462187 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:11.462159    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w44sl" event={"ID":"c65c2a43-435b-4da9-8d1a-9d2a550449ab","Type":"ContainerStarted","Data":"28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51"}
Jul 24 16:56:11.475622 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:11.475577    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-drbcn" podStartSLOduration=2.155241776 podStartE2EDuration="5.475563399s" totalImagesPullingTime="3.320321623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:56:11.474389634 +0000 UTC m=+778.234198634" watchObservedRunningTime="2026-07-24 16:56:11.475563399 +0000 UTC m=+778.235372438"
Jul 24 16:56:11.488924 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:11.488877    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-w44sl" podStartSLOduration=1.9391357660000001 podStartE2EDuration="5.488861153s" totalImagesPullingTime="3.549725387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:56:11.487908807 +0000 UTC m=+778.247717807" watchObservedRunningTime="2026-07-24 16:56:11.488861153 +0000 UTC m=+778.248670151"
Jul 24 16:56:11.512693 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:11.512653    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-w44sl"]
Jul 24 16:56:13.471661 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:13.471592    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-w44sl" podUID="c65c2a43-435b-4da9-8d1a-9d2a550449ab" containerName="authorino" containerID="cri-o://28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51" gracePeriod=30
Jul 24 16:56:13.722233 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:13.722174    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w44sl"
Jul 24 16:56:13.756490 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:13.756459    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx5cm\" (UniqueName: \"kubernetes.io/projected/c65c2a43-435b-4da9-8d1a-9d2a550449ab-kube-api-access-fx5cm\") pod \"c65c2a43-435b-4da9-8d1a-9d2a550449ab\" (UID: \"c65c2a43-435b-4da9-8d1a-9d2a550449ab\") "
Jul 24 16:56:13.758521 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:13.758493    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c65c2a43-435b-4da9-8d1a-9d2a550449ab-kube-api-access-fx5cm" (OuterVolumeSpecName: "kube-api-access-fx5cm") pod "c65c2a43-435b-4da9-8d1a-9d2a550449ab" (UID: "c65c2a43-435b-4da9-8d1a-9d2a550449ab"). InnerVolumeSpecName "kube-api-access-fx5cm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:56:13.857132 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:13.857095    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx5cm\" (UniqueName: \"kubernetes.io/projected/c65c2a43-435b-4da9-8d1a-9d2a550449ab-kube-api-access-fx5cm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:56:14.482523 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.482476    2569 generic.go:356] "Generic (PLEG): container finished" podID="c65c2a43-435b-4da9-8d1a-9d2a550449ab" containerID="28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51" exitCode=0
Jul 24 16:56:14.482919 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.482548    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w44sl"
Jul 24 16:56:14.482919 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.482562    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w44sl" event={"ID":"c65c2a43-435b-4da9-8d1a-9d2a550449ab","Type":"ContainerDied","Data":"28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51"}
Jul 24 16:56:14.482919 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.482600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w44sl" event={"ID":"c65c2a43-435b-4da9-8d1a-9d2a550449ab","Type":"ContainerDied","Data":"fb5e77e4df83a619f0c32e25fb8356a35a493627d5421af0d7f32a529552dff9"}
Jul 24 16:56:14.482919 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.482618    2569 scope.go:117] "RemoveContainer" containerID="28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51"
Jul 24 16:56:14.491241 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.491226    2569 scope.go:117] "RemoveContainer" containerID="28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51"
Jul 24 16:56:14.491524 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:56:14.491504    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51\": container with ID starting with 28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51 not found: ID does not exist" containerID="28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51"
Jul 24 16:56:14.491592 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.491531    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51"} err="failed to get container status \"28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51\": rpc error: code = NotFound desc = could not find container \"28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51\": container with ID starting with 28ed9a32a02bd5ba451acf196b0a6cfb2a1e38154daa380c80463035645b1a51 not found: ID does not exist"
Jul 24 16:56:14.502587 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.502563    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-w44sl"]
Jul 24 16:56:14.505292 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:14.505270    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-w44sl"]
Jul 24 16:56:15.629489 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:15.629458    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c65c2a43-435b-4da9-8d1a-9d2a550449ab" path="/var/lib/kubelet/pods/c65c2a43-435b-4da9-8d1a-9d2a550449ab/volumes"
Jul 24 16:56:35.716552 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.716454    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k6kzs"]
Jul 24 16:56:35.717001 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.716842    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c65c2a43-435b-4da9-8d1a-9d2a550449ab" containerName="authorino"
Jul 24 16:56:35.717001 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.716857    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c65c2a43-435b-4da9-8d1a-9d2a550449ab" containerName="authorino"
Jul 24 16:56:35.717001 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.716927    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c65c2a43-435b-4da9-8d1a-9d2a550449ab" containerName="authorino"
Jul 24 16:56:35.722764 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.722744    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k6kzs"
Jul 24 16:56:35.725983 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.725953    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k6kzs"]
Jul 24 16:56:35.815623 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.815592    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfvnm\" (UniqueName: \"kubernetes.io/projected/a0996966-7cce-41d0-ab34-28d5912a7410-kube-api-access-xfvnm\") pod \"authorino-5d4b67c446-k6kzs\" (UID: \"a0996966-7cce-41d0-ab34-28d5912a7410\") " pod="kuadrant-system/authorino-5d4b67c446-k6kzs"
Jul 24 16:56:35.891533 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.891498    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k6kzs"]
Jul 24 16:56:35.891765 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:56:35.891747    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xfvnm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-k6kzs" podUID="a0996966-7cce-41d0-ab34-28d5912a7410"
Jul 24 16:56:35.916203 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.916174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfvnm\" (UniqueName: \"kubernetes.io/projected/a0996966-7cce-41d0-ab34-28d5912a7410-kube-api-access-xfvnm\") pod \"authorino-5d4b67c446-k6kzs\" (UID: \"a0996966-7cce-41d0-ab34-28d5912a7410\") " pod="kuadrant-system/authorino-5d4b67c446-k6kzs"
Jul 24 16:56:35.918488 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.918462    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-vkjj4"]
Jul 24 16:56:35.921586 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.921569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:35.923694 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.923674    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 16:56:35.924768 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.924748    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfvnm\" (UniqueName: \"kubernetes.io/projected/a0996966-7cce-41d0-ab34-28d5912a7410-kube-api-access-xfvnm\") pod \"authorino-5d4b67c446-k6kzs\" (UID: \"a0996966-7cce-41d0-ab34-28d5912a7410\") " pod="kuadrant-system/authorino-5d4b67c446-k6kzs"
Jul 24 16:56:35.930043 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.930021    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-vkjj4"]
Jul 24 16:56:35.965076 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.965033    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-vkjj4"]
Jul 24 16:56:35.965474 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:56:35.965443    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fxxft tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-fxxft tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-vkjj4" podUID="5ef0bceb-420c-4cc5-9981-de52d9f03767"
Jul 24 16:56:35.991798 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.991720    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64675f474d-gfq7l"]
Jul 24 16:56:35.995133 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:35.995112    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.000961 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.000937    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64675f474d-gfq7l"]
Jul 24 16:56:36.017309 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.017280    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-tls-cert\") pod \"authorino-64675f474d-gfq7l\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") " pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.017475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.017314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxxft\" (UniqueName: \"kubernetes.io/projected/5ef0bceb-420c-4cc5-9981-de52d9f03767-kube-api-access-fxxft\") pod \"authorino-76877994-vkjj4\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") " pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.017475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.017387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ef0bceb-420c-4cc5-9981-de52d9f03767-tls-cert\") pod \"authorino-76877994-vkjj4\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") " pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.017475 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.017434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bn6\" (UniqueName: \"kubernetes.io/projected/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-kube-api-access-g5bn6\") pod \"authorino-64675f474d-gfq7l\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") " pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.117945 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.117912    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-tls-cert\") pod \"authorino-64675f474d-gfq7l\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") " pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.117945 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.117949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxxft\" (UniqueName: \"kubernetes.io/projected/5ef0bceb-420c-4cc5-9981-de52d9f03767-kube-api-access-fxxft\") pod \"authorino-76877994-vkjj4\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") " pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.118192 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.117971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ef0bceb-420c-4cc5-9981-de52d9f03767-tls-cert\") pod \"authorino-76877994-vkjj4\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") " pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.118192 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.117995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bn6\" (UniqueName: \"kubernetes.io/projected/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-kube-api-access-g5bn6\") pod \"authorino-64675f474d-gfq7l\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") " pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.120402 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.120375    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-tls-cert\") pod \"authorino-64675f474d-gfq7l\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") " pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.120555 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.120536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ef0bceb-420c-4cc5-9981-de52d9f03767-tls-cert\") pod \"authorino-76877994-vkjj4\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") " pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.126434 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.126404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bn6\" (UniqueName: \"kubernetes.io/projected/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-kube-api-access-g5bn6\") pod \"authorino-64675f474d-gfq7l\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") " pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.126434 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.126429    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxxft\" (UniqueName: \"kubernetes.io/projected/5ef0bceb-420c-4cc5-9981-de52d9f03767-kube-api-access-fxxft\") pod \"authorino-76877994-vkjj4\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") " pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.304307 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.304227    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:56:36.435612 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.435565    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64675f474d-gfq7l"]
Jul 24 16:56:36.438883 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:56:36.438856    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode05ca8e5_7d50_43d6_831c_01bd2c7abb17.slice/crio-8d2ccc5f81b086fde131e12cbf3d8d25d853d84a11b241852876e95cbd042a3d WatchSource:0}: Error finding container 8d2ccc5f81b086fde131e12cbf3d8d25d853d84a11b241852876e95cbd042a3d: Status 404 returned error can't find the container with id 8d2ccc5f81b086fde131e12cbf3d8d25d853d84a11b241852876e95cbd042a3d
Jul 24 16:56:36.581109 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.581080    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.581109 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.581080    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64675f474d-gfq7l" event={"ID":"e05ca8e5-7d50-43d6-831c-01bd2c7abb17","Type":"ContainerStarted","Data":"8d2ccc5f81b086fde131e12cbf3d8d25d853d84a11b241852876e95cbd042a3d"}
Jul 24 16:56:36.581317 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.581228    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k6kzs"
Jul 24 16:56:36.589398 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.589379    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:36.591426 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.591410    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k6kzs"
Jul 24 16:56:36.622575 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.622553    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxxft\" (UniqueName: \"kubernetes.io/projected/5ef0bceb-420c-4cc5-9981-de52d9f03767-kube-api-access-fxxft\") pod \"5ef0bceb-420c-4cc5-9981-de52d9f03767\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") "
Jul 24 16:56:36.622692 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.622588    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfvnm\" (UniqueName: \"kubernetes.io/projected/a0996966-7cce-41d0-ab34-28d5912a7410-kube-api-access-xfvnm\") pod \"a0996966-7cce-41d0-ab34-28d5912a7410\" (UID: \"a0996966-7cce-41d0-ab34-28d5912a7410\") "
Jul 24 16:56:36.622692 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.622626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ef0bceb-420c-4cc5-9981-de52d9f03767-tls-cert\") pod \"5ef0bceb-420c-4cc5-9981-de52d9f03767\" (UID: \"5ef0bceb-420c-4cc5-9981-de52d9f03767\") "
Jul 24 16:56:36.624656 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.624631    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef0bceb-420c-4cc5-9981-de52d9f03767-kube-api-access-fxxft" (OuterVolumeSpecName: "kube-api-access-fxxft") pod "5ef0bceb-420c-4cc5-9981-de52d9f03767" (UID: "5ef0bceb-420c-4cc5-9981-de52d9f03767"). InnerVolumeSpecName "kube-api-access-fxxft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:56:36.624768 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.624650    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ef0bceb-420c-4cc5-9981-de52d9f03767-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5ef0bceb-420c-4cc5-9981-de52d9f03767" (UID: "5ef0bceb-420c-4cc5-9981-de52d9f03767"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:56:36.624768 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.624739    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0996966-7cce-41d0-ab34-28d5912a7410-kube-api-access-xfvnm" (OuterVolumeSpecName: "kube-api-access-xfvnm") pod "a0996966-7cce-41d0-ab34-28d5912a7410" (UID: "a0996966-7cce-41d0-ab34-28d5912a7410"). InnerVolumeSpecName "kube-api-access-xfvnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:56:36.723761 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.723708    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxxft\" (UniqueName: \"kubernetes.io/projected/5ef0bceb-420c-4cc5-9981-de52d9f03767-kube-api-access-fxxft\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:56:36.723761 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.723740    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfvnm\" (UniqueName: \"kubernetes.io/projected/a0996966-7cce-41d0-ab34-28d5912a7410-kube-api-access-xfvnm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:56:36.723761 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:36.723755    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ef0bceb-420c-4cc5-9981-de52d9f03767-tls-cert\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:56:37.587930 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.587902    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-k6kzs"
Jul 24 16:56:37.588108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.587941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64675f474d-gfq7l" event={"ID":"e05ca8e5-7d50-43d6-831c-01bd2c7abb17","Type":"ContainerStarted","Data":"443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d"}
Jul 24 16:56:37.588108 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.588083    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vkjj4"
Jul 24 16:56:37.606417 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.606380    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64675f474d-gfq7l" podStartSLOduration=2.269445191 podStartE2EDuration="2.60636579s" totalImagesPullingTime="336.920599ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:56:37.605724678 +0000 UTC m=+804.365533678" watchObservedRunningTime="2026-07-24 16:56:37.60636579 +0000 UTC m=+804.366174786"
Jul 24 16:56:37.631309 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.631280    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-vkjj4"]
Jul 24 16:56:37.637289 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.637266    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-drbcn"]
Jul 24 16:56:37.637665 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.637637    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-drbcn" podUID="eacfe1eb-901f-4e8e-971a-8621f9a23525" containerName="authorino" containerID="cri-o://7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad" gracePeriod=30
Jul 24 16:56:37.639571 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.639549    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-vkjj4"]
Jul 24 16:56:37.661295 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.661272    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k6kzs"]
Jul 24 16:56:37.665155 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.665132    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-k6kzs"]
Jul 24 16:56:37.888297 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.888271    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-drbcn"
Jul 24 16:56:37.933391 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.933366    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs5hj\" (UniqueName: \"kubernetes.io/projected/eacfe1eb-901f-4e8e-971a-8621f9a23525-kube-api-access-fs5hj\") pod \"eacfe1eb-901f-4e8e-971a-8621f9a23525\" (UID: \"eacfe1eb-901f-4e8e-971a-8621f9a23525\") "
Jul 24 16:56:37.935358 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:37.935325    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eacfe1eb-901f-4e8e-971a-8621f9a23525-kube-api-access-fs5hj" (OuterVolumeSpecName: "kube-api-access-fs5hj") pod "eacfe1eb-901f-4e8e-971a-8621f9a23525" (UID: "eacfe1eb-901f-4e8e-971a-8621f9a23525"). InnerVolumeSpecName "kube-api-access-fs5hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:56:38.034957 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.034924    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs5hj\" (UniqueName: \"kubernetes.io/projected/eacfe1eb-901f-4e8e-971a-8621f9a23525-kube-api-access-fs5hj\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:56:38.593618 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.593583    2569 generic.go:356] "Generic (PLEG): container finished" podID="eacfe1eb-901f-4e8e-971a-8621f9a23525" containerID="7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad" exitCode=0
Jul 24 16:56:38.593797 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.593629    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-drbcn"
Jul 24 16:56:38.593797 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.593663    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-drbcn" event={"ID":"eacfe1eb-901f-4e8e-971a-8621f9a23525","Type":"ContainerDied","Data":"7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad"}
Jul 24 16:56:38.593797 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.593699    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-drbcn" event={"ID":"eacfe1eb-901f-4e8e-971a-8621f9a23525","Type":"ContainerDied","Data":"7b2a631306f15ffd1ba215bfab7bc024a6a730622f8ecad5ed345ea919da89db"}
Jul 24 16:56:38.593797 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.593716    2569 scope.go:117] "RemoveContainer" containerID="7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad"
Jul 24 16:56:38.602533 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.602511    2569 scope.go:117] "RemoveContainer" containerID="7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad"
Jul 24 16:56:38.602775 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:56:38.602756    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad\": container with ID starting with 7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad not found: ID does not exist" containerID="7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad"
Jul 24 16:56:38.602824 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.602785    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad"} err="failed to get container status \"7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad\": rpc error: code = NotFound desc = could not find container \"7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad\": container with ID starting with 7fb57ad60ef6b14ac674b20efdd98737e05fe0788531921e22eee8f5d948ecad not found: ID does not exist"
Jul 24 16:56:38.614607 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.614582    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-drbcn"]
Jul 24 16:56:38.618214 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:38.618194    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-drbcn"]
Jul 24 16:56:39.630019 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:39.629986    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ef0bceb-420c-4cc5-9981-de52d9f03767" path="/var/lib/kubelet/pods/5ef0bceb-420c-4cc5-9981-de52d9f03767/volumes"
Jul 24 16:56:39.630399 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:39.630210    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0996966-7cce-41d0-ab34-28d5912a7410" path="/var/lib/kubelet/pods/a0996966-7cce-41d0-ab34-28d5912a7410/volumes"
Jul 24 16:56:39.630452 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:56:39.630422    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eacfe1eb-901f-4e8e-971a-8621f9a23525" path="/var/lib/kubelet/pods/eacfe1eb-901f-4e8e-971a-8621f9a23525/volumes"
Jul 24 16:57:41.392948 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.392919    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"]
Jul 24 16:57:41.393433 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.393246    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eacfe1eb-901f-4e8e-971a-8621f9a23525" containerName="authorino"
Jul 24 16:57:41.393433 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.393256    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eacfe1eb-901f-4e8e-971a-8621f9a23525" containerName="authorino"
Jul 24 16:57:41.393433 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.393307    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eacfe1eb-901f-4e8e-971a-8621f9a23525" containerName="authorino"
Jul 24 16:57:41.396252 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.396232    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.398650 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.398632    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 16:57:41.399500 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.399485    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 16:57:41.399828 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.399806    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s45jh\""
Jul 24 16:57:41.399828 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.399822    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 24 16:57:41.405308 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.405284    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"]
Jul 24 16:57:41.491291 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.491255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.491484 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.491306    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4jhs\" (UniqueName: \"kubernetes.io/projected/f3e85495-a01b-406d-9294-ecd3a8096730-kube-api-access-h4jhs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.491484 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.491364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.491484 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.491387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3e85495-a01b-406d-9294-ecd3a8096730-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.491484 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.491415    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.491484 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.491458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.591972 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.591933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4jhs\" (UniqueName: \"kubernetes.io/projected/f3e85495-a01b-406d-9294-ecd3a8096730-kube-api-access-h4jhs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.591972 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.591976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.592222 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.591996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3e85495-a01b-406d-9294-ecd3a8096730-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.592222 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.592038    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.592222 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.592058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.592222 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.592110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.592507 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.592469    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.592613 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.592508    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.592613 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.592520    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.594435 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.594414    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3e85495-a01b-406d-9294-ecd3a8096730-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.594688 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.594669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3e85495-a01b-406d-9294-ecd3a8096730-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.599561 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.599539    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4jhs\" (UniqueName: \"kubernetes.io/projected/f3e85495-a01b-406d-9294-ecd3a8096730-kube-api-access-h4jhs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd\" (UID: \"f3e85495-a01b-406d-9294-ecd3a8096730\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.707993 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.707907    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:41.838882 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.838835    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"]
Jul 24 16:57:41.841191 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:57:41.841165    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e85495_a01b_406d_9294_ecd3a8096730.slice/crio-1c78ce0749b55d8849179ae5a5e2c4d6e0cd4d38afd41d7fca4dfa9e4bd8ffe4 WatchSource:0}: Error finding container 1c78ce0749b55d8849179ae5a5e2c4d6e0cd4d38afd41d7fca4dfa9e4bd8ffe4: Status 404 returned error can't find the container with id 1c78ce0749b55d8849179ae5a5e2c4d6e0cd4d38afd41d7fca4dfa9e4bd8ffe4
Jul 24 16:57:41.868946 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:41.868921    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd" event={"ID":"f3e85495-a01b-406d-9294-ecd3a8096730","Type":"ContainerStarted","Data":"1c78ce0749b55d8849179ae5a5e2c4d6e0cd4d38afd41d7fca4dfa9e4bd8ffe4"}
Jul 24 16:57:43.878860 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:43.878820    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd" event={"ID":"f3e85495-a01b-406d-9294-ecd3a8096730","Type":"ContainerStarted","Data":"684cd4f4b5c101640c53b97a80f4e975399246b6307644d425aba38d24fd78fe"}
Jul 24 16:57:43.902353 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:43.902277    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd" podStartSLOduration=1.583412156 podStartE2EDuration="2.90226231s" totalImagesPullingTime="1.318850154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:57:43.900630342 +0000 UTC m=+870.660439365" watchObservedRunningTime="2026-07-24 16:57:43.90226231 +0000 UTC m=+870.662071309"
Jul 24 16:57:51.708206 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:51.708172    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:51.708729 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:51.708240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:51.720818 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:51.720794    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:57:51.924760 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:57:51.924730    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd"
Jul 24 16:58:18.683539 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.683507    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"]
Jul 24 16:58:18.686886 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.686867    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.689320 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.689299    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 16:58:18.698916 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.698895    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"]
Jul 24 16:58:18.816131 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.816093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.816131 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.816137    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.816401 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.816155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjdw9\" (UniqueName: \"kubernetes.io/projected/9bd74c75-ccdf-4746-a7e0-a459acf343d2-kube-api-access-vjdw9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.816401 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.816187    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bd74c75-ccdf-4746-a7e0-a459acf343d2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.816401 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.816219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.816401 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.816258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.916861 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.916821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.916878    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.916918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.916941    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjdw9\" (UniqueName: \"kubernetes.io/projected/9bd74c75-ccdf-4746-a7e0-a459acf343d2-kube-api-access-vjdw9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.916971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bd74c75-ccdf-4746-a7e0-a459acf343d2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917037 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.917022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917423 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.917389    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917423 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.917416    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.917423 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.917421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.919132 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.919113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bd74c75-ccdf-4746-a7e0-a459acf343d2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.919480 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.919459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bd74c75-ccdf-4746-a7e0-a459acf343d2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.925897 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.925873    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjdw9\" (UniqueName: \"kubernetes.io/projected/9bd74c75-ccdf-4746-a7e0-a459acf343d2-kube-api-access-vjdw9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln\" (UID: \"9bd74c75-ccdf-4746-a7e0-a459acf343d2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:18.997900 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:18.997827    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:19.132432 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:19.132407    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"]
Jul 24 16:58:19.135154 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:58:19.135116    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd74c75_ccdf_4746_a7e0_a459acf343d2.slice/crio-1632ec37098f36af383a76cb1904addcaf0130e93712f4d3e5f7e9d90e2a770d WatchSource:0}: Error finding container 1632ec37098f36af383a76cb1904addcaf0130e93712f4d3e5f7e9d90e2a770d: Status 404 returned error can't find the container with id 1632ec37098f36af383a76cb1904addcaf0130e93712f4d3e5f7e9d90e2a770d
Jul 24 16:58:20.035514 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:20.035479    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln" event={"ID":"9bd74c75-ccdf-4746-a7e0-a459acf343d2","Type":"ContainerStarted","Data":"eae6b779390ddf5fbfa68262ed12650a63b21dfdfbefd9930364aaf09f5ca91d"}
Jul 24 16:58:20.035937 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:20.035534    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln" event={"ID":"9bd74c75-ccdf-4746-a7e0-a459acf343d2","Type":"ContainerStarted","Data":"1632ec37098f36af383a76cb1904addcaf0130e93712f4d3e5f7e9d90e2a770d"}
Jul 24 16:58:20.053477 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:20.053427    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln" podStartSLOduration=1.804165493 podStartE2EDuration="2.053411314s" totalImagesPullingTime="249.245821ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:58:20.052595172 +0000 UTC m=+906.812404172" watchObservedRunningTime="2026-07-24 16:58:20.053411314 +0000 UTC m=+906.813220313"
Jul 24 16:58:28.998271 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:28.998220    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:28.998671 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:28.998321    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:29.011302 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:29.011273    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:29.093129 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:29.093094    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln"
Jul 24 16:58:33.887612 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.887583    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"]
Jul 24 16:58:33.890921 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.890906    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:33.893182 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.893162    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 16:58:33.900583 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.900561    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"]
Jul 24 16:58:33.957279 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.957247    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:33.957453 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.957292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:33.957453 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.957320    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:33.957453 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.957413    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:33.957580 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.957451    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsjnr\" (UniqueName: \"kubernetes.io/projected/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-kube-api-access-wsjnr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:33.957580 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:33.957544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.058995 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.058954    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.058995 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsjnr\" (UniqueName: \"kubernetes.io/projected/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-kube-api-access-wsjnr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.059252 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.059252 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.059252 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.059252 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059237    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.059538 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.059588 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059566    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.059628 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.059593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.061268 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.061241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.061577 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.061552    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.066316 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.066298    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsjnr\" (UniqueName: \"kubernetes.io/projected/039ce2dd-0db5-4c1b-aa62-b5032aa1f119-kube-api-access-wsjnr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2\" (UID: \"039ce2dd-0db5-4c1b-aa62-b5032aa1f119\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.201080 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.200978    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:34.327068 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:34.326995    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"]
Jul 24 16:58:34.330171 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:58:34.330142    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod039ce2dd_0db5_4c1b_aa62_b5032aa1f119.slice/crio-840b7d428df2ba0cc0836531aa1daadb3db9f85417cdeb33df2cb9793b2d9e5d WatchSource:0}: Error finding container 840b7d428df2ba0cc0836531aa1daadb3db9f85417cdeb33df2cb9793b2d9e5d: Status 404 returned error can't find the container with id 840b7d428df2ba0cc0836531aa1daadb3db9f85417cdeb33df2cb9793b2d9e5d
Jul 24 16:58:35.111248 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:35.111213    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2" event={"ID":"039ce2dd-0db5-4c1b-aa62-b5032aa1f119","Type":"ContainerStarted","Data":"e90dfb9064dac0a994ff944d3ae631264c8d6f06d5f3d6e5254a78d1659f0ac7"}
Jul 24 16:58:35.111711 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:35.111258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2" event={"ID":"039ce2dd-0db5-4c1b-aa62-b5032aa1f119","Type":"ContainerStarted","Data":"840b7d428df2ba0cc0836531aa1daadb3db9f85417cdeb33df2cb9793b2d9e5d"}
Jul 24 16:58:35.129672 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:35.129623    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2" podStartSLOduration=1.736438604 podStartE2EDuration="2.129607569s" totalImagesPullingTime="393.168965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:58:35.127608739 +0000 UTC m=+921.887417739" watchObservedRunningTime="2026-07-24 16:58:35.129607569 +0000 UTC m=+921.889416568"
Jul 24 16:58:44.201243 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:44.201199    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:44.201726 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:44.201262    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:44.214355 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:44.214315    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:58:45.167509 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:58:45.167478    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2"
Jul 24 16:59:21.646845 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.646812    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b988cc965-rpr8j"]
Jul 24 16:59:21.650304 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.650287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:21.661595 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.661573    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b988cc965-rpr8j"]
Jul 24 16:59:21.773189 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.773147    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkplx\" (UniqueName: \"kubernetes.io/projected/627c3596-6db9-47f3-b592-8cd25a2556b2-kube-api-access-fkplx\") pod \"authorino-5b988cc965-rpr8j\" (UID: \"627c3596-6db9-47f3-b592-8cd25a2556b2\") " pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:21.773189 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.773189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/627c3596-6db9-47f3-b592-8cd25a2556b2-tls-cert\") pod \"authorino-5b988cc965-rpr8j\" (UID: \"627c3596-6db9-47f3-b592-8cd25a2556b2\") " pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:21.874503 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.874466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkplx\" (UniqueName: \"kubernetes.io/projected/627c3596-6db9-47f3-b592-8cd25a2556b2-kube-api-access-fkplx\") pod \"authorino-5b988cc965-rpr8j\" (UID: \"627c3596-6db9-47f3-b592-8cd25a2556b2\") " pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:21.874503 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.874506    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/627c3596-6db9-47f3-b592-8cd25a2556b2-tls-cert\") pod \"authorino-5b988cc965-rpr8j\" (UID: \"627c3596-6db9-47f3-b592-8cd25a2556b2\") " pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:21.876903 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.876882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/627c3596-6db9-47f3-b592-8cd25a2556b2-tls-cert\") pod \"authorino-5b988cc965-rpr8j\" (UID: \"627c3596-6db9-47f3-b592-8cd25a2556b2\") " pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:21.887592 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.887551    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkplx\" (UniqueName: \"kubernetes.io/projected/627c3596-6db9-47f3-b592-8cd25a2556b2-kube-api-access-fkplx\") pod \"authorino-5b988cc965-rpr8j\" (UID: \"627c3596-6db9-47f3-b592-8cd25a2556b2\") " pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:21.959445 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:21.959367    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b988cc965-rpr8j"
Jul 24 16:59:22.092394 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:22.092185    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b988cc965-rpr8j"]
Jul 24 16:59:22.094540 ip-10-0-135-204 kubenswrapper[2569]: W0724 16:59:22.094508    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod627c3596_6db9_47f3_b592_8cd25a2556b2.slice/crio-7573699bc397593c4923192e490450aa6741a29ed66cf27af28caf102a514a7e WatchSource:0}: Error finding container 7573699bc397593c4923192e490450aa6741a29ed66cf27af28caf102a514a7e: Status 404 returned error can't find the container with id 7573699bc397593c4923192e490450aa6741a29ed66cf27af28caf102a514a7e
Jul 24 16:59:22.320917 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:22.320879    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b988cc965-rpr8j" event={"ID":"627c3596-6db9-47f3-b592-8cd25a2556b2","Type":"ContainerStarted","Data":"7573699bc397593c4923192e490450aa6741a29ed66cf27af28caf102a514a7e"}
Jul 24 16:59:23.330366 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.330314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b988cc965-rpr8j" event={"ID":"627c3596-6db9-47f3-b592-8cd25a2556b2","Type":"ContainerStarted","Data":"80a45fa01d37f0c00e570416d9f001f0e9d4365de9fb3e400fd68352d0bbc1ea"}
Jul 24 16:59:23.348727 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.348670    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b988cc965-rpr8j" podStartSLOduration=1.9006844680000001 podStartE2EDuration="2.348653587s" totalImagesPullingTime="447.969119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:59:23.345757228 +0000 UTC m=+970.105566228" watchObservedRunningTime="2026-07-24 16:59:23.348653587 +0000 UTC m=+970.108462585"
Jul 24 16:59:23.377544 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.377491    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64675f474d-gfq7l"]
Jul 24 16:59:23.377785 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.377751    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-64675f474d-gfq7l" podUID="e05ca8e5-7d50-43d6-831c-01bd2c7abb17" containerName="authorino" containerID="cri-o://443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d" gracePeriod=30
Jul 24 16:59:23.639525 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.639497    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:59:23.694572 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.694536    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-tls-cert\") pod \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") "
Jul 24 16:59:23.695225 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.695204    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5bn6\" (UniqueName: \"kubernetes.io/projected/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-kube-api-access-g5bn6\") pod \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\" (UID: \"e05ca8e5-7d50-43d6-831c-01bd2c7abb17\") "
Jul 24 16:59:23.698165 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.698126    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-kube-api-access-g5bn6" (OuterVolumeSpecName: "kube-api-access-g5bn6") pod "e05ca8e5-7d50-43d6-831c-01bd2c7abb17" (UID: "e05ca8e5-7d50-43d6-831c-01bd2c7abb17"). InnerVolumeSpecName "kube-api-access-g5bn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:59:23.712179 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.712146    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e05ca8e5-7d50-43d6-831c-01bd2c7abb17" (UID: "e05ca8e5-7d50-43d6-831c-01bd2c7abb17"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:59:23.796045 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.796010    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5bn6\" (UniqueName: \"kubernetes.io/projected/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-kube-api-access-g5bn6\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:59:23.796045 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:23.796041    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e05ca8e5-7d50-43d6-831c-01bd2c7abb17-tls-cert\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 16:59:24.336204 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.336167    2569 generic.go:356] "Generic (PLEG): container finished" podID="e05ca8e5-7d50-43d6-831c-01bd2c7abb17" containerID="443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d" exitCode=0
Jul 24 16:59:24.336709 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.336219    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64675f474d-gfq7l"
Jul 24 16:59:24.336709 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.336261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64675f474d-gfq7l" event={"ID":"e05ca8e5-7d50-43d6-831c-01bd2c7abb17","Type":"ContainerDied","Data":"443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d"}
Jul 24 16:59:24.336709 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.336311    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64675f474d-gfq7l" event={"ID":"e05ca8e5-7d50-43d6-831c-01bd2c7abb17","Type":"ContainerDied","Data":"8d2ccc5f81b086fde131e12cbf3d8d25d853d84a11b241852876e95cbd042a3d"}
Jul 24 16:59:24.336709 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.336360    2569 scope.go:117] "RemoveContainer" containerID="443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d"
Jul 24 16:59:24.346830 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.346815    2569 scope.go:117] "RemoveContainer" containerID="443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d"
Jul 24 16:59:24.347102 ip-10-0-135-204 kubenswrapper[2569]: E0724 16:59:24.347084    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d\": container with ID starting with 443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d not found: ID does not exist" containerID="443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d"
Jul 24 16:59:24.347154 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.347113    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d"} err="failed to get container status \"443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d\": rpc error: code = NotFound desc = could not find container \"443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d\": container with ID starting with 443d614179936a257b6e68d29b8495e51bd7e4227eec6f79473c7ead906de18d not found: ID does not exist"
Jul 24 16:59:24.362082 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.362050    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64675f474d-gfq7l"]
Jul 24 16:59:24.374894 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:24.374860    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64675f474d-gfq7l"]
Jul 24 16:59:25.629752 ip-10-0-135-204 kubenswrapper[2569]: I0724 16:59:25.629721    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e05ca8e5-7d50-43d6-831c-01bd2c7abb17" path="/var/lib/kubelet/pods/e05ca8e5-7d50-43d6-831c-01bd2c7abb17/volumes"
Jul 24 17:06:11.587284 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:11.587235    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"]
Jul 24 17:06:11.587807 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:11.587472    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4" podUID="a5649270-b49e-4015-bed3-3f7f4ac4196d" containerName="manager" containerID="cri-o://ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98" gracePeriod=10
Jul 24 17:06:15.042228 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.042199    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 17:06:15.110377 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.110343    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5649270-b49e-4015-bed3-3f7f4ac4196d-extensions-socket-volume\") pod \"a5649270-b49e-4015-bed3-3f7f4ac4196d\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") "
Jul 24 17:06:15.110546 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.110475    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzc82\" (UniqueName: \"kubernetes.io/projected/a5649270-b49e-4015-bed3-3f7f4ac4196d-kube-api-access-jzc82\") pod \"a5649270-b49e-4015-bed3-3f7f4ac4196d\" (UID: \"a5649270-b49e-4015-bed3-3f7f4ac4196d\") "
Jul 24 17:06:15.110752 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.110720    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5649270-b49e-4015-bed3-3f7f4ac4196d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a5649270-b49e-4015-bed3-3f7f4ac4196d" (UID: "a5649270-b49e-4015-bed3-3f7f4ac4196d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:06:15.112553 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.112530    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5649270-b49e-4015-bed3-3f7f4ac4196d-kube-api-access-jzc82" (OuterVolumeSpecName: "kube-api-access-jzc82") pod "a5649270-b49e-4015-bed3-3f7f4ac4196d" (UID: "a5649270-b49e-4015-bed3-3f7f4ac4196d"). InnerVolumeSpecName "kube-api-access-jzc82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:06:15.184262 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.184159    2569 generic.go:356] "Generic (PLEG): container finished" podID="a5649270-b49e-4015-bed3-3f7f4ac4196d" containerID="ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98" exitCode=0
Jul 24 17:06:15.184262 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.184205    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4" event={"ID":"a5649270-b49e-4015-bed3-3f7f4ac4196d","Type":"ContainerDied","Data":"ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98"}
Jul 24 17:06:15.184262 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.184210    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"
Jul 24 17:06:15.184563 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.184278    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4" event={"ID":"a5649270-b49e-4015-bed3-3f7f4ac4196d","Type":"ContainerDied","Data":"99ab7b3c9dcf6a2768112102fbbcc41d79353a908fd875ac1e9114c229f3fb37"}
Jul 24 17:06:15.184563 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.184304    2569 scope.go:117] "RemoveContainer" containerID="ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98"
Jul 24 17:06:15.194199 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.194181    2569 scope.go:117] "RemoveContainer" containerID="ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98"
Jul 24 17:06:15.194549 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:06:15.194518    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98\": container with ID starting with ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98 not found: ID does not exist" containerID="ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98"
Jul 24 17:06:15.194629 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.194561    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98"} err="failed to get container status \"ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98\": rpc error: code = NotFound desc = could not find container \"ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98\": container with ID starting with ff10309ef0e77853f97236478600300d51c21cf49d30a698eaf366f391692d98 not found: ID does not exist"
Jul 24 17:06:15.205778 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.205748    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"]
Jul 24 17:06:15.207282 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.207264    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ptxs4"]
Jul 24 17:06:15.212446 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.212425    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5649270-b49e-4015-bed3-3f7f4ac4196d-extensions-socket-volume\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:06:15.212446 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.212447    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzc82\" (UniqueName: \"kubernetes.io/projected/a5649270-b49e-4015-bed3-3f7f4ac4196d-kube-api-access-jzc82\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:06:15.629501 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:06:15.629473    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5649270-b49e-4015-bed3-3f7f4ac4196d" path="/var/lib/kubelet/pods/a5649270-b49e-4015-bed3-3f7f4ac4196d/volumes"
Jul 24 17:07:12.748987 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.748910    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"]
Jul 24 17:07:12.749432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.749273    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5649270-b49e-4015-bed3-3f7f4ac4196d" containerName="manager"
Jul 24 17:07:12.749432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.749285    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5649270-b49e-4015-bed3-3f7f4ac4196d" containerName="manager"
Jul 24 17:07:12.749432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.749295    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05ca8e5-7d50-43d6-831c-01bd2c7abb17" containerName="authorino"
Jul 24 17:07:12.749432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.749301    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05ca8e5-7d50-43d6-831c-01bd2c7abb17" containerName="authorino"
Jul 24 17:07:12.749432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.749372    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05ca8e5-7d50-43d6-831c-01bd2c7abb17" containerName="authorino"
Jul 24 17:07:12.749432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.749383    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5649270-b49e-4015-bed3-3f7f4ac4196d" containerName="manager"
Jul 24 17:07:12.752313 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.752296    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:12.754631 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.754606    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f9j46\""
Jul 24 17:07:12.763536 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.763513    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"]
Jul 24 17:07:12.808035 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.807997    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgdxz\" (UniqueName: \"kubernetes.io/projected/d99834be-a9c5-4309-86ed-f5d047e97c6f-kube-api-access-hgdxz\") pod \"kuadrant-operator-controller-manager-86878c769f-2r7cw\" (UID: \"d99834be-a9c5-4309-86ed-f5d047e97c6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:12.808196 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.808109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d99834be-a9c5-4309-86ed-f5d047e97c6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2r7cw\" (UID: \"d99834be-a9c5-4309-86ed-f5d047e97c6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:12.908727 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.908685    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d99834be-a9c5-4309-86ed-f5d047e97c6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2r7cw\" (UID: \"d99834be-a9c5-4309-86ed-f5d047e97c6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:12.908899 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.908751    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgdxz\" (UniqueName: \"kubernetes.io/projected/d99834be-a9c5-4309-86ed-f5d047e97c6f-kube-api-access-hgdxz\") pod \"kuadrant-operator-controller-manager-86878c769f-2r7cw\" (UID: \"d99834be-a9c5-4309-86ed-f5d047e97c6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:12.909094 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.909077    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d99834be-a9c5-4309-86ed-f5d047e97c6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2r7cw\" (UID: \"d99834be-a9c5-4309-86ed-f5d047e97c6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:12.920114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:12.920087    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgdxz\" (UniqueName: \"kubernetes.io/projected/d99834be-a9c5-4309-86ed-f5d047e97c6f-kube-api-access-hgdxz\") pod \"kuadrant-operator-controller-manager-86878c769f-2r7cw\" (UID: \"d99834be-a9c5-4309-86ed-f5d047e97c6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:13.062886 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:13.062788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:13.199192 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:13.199169    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"]
Jul 24 17:07:13.201987 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:07:13.201957    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99834be_a9c5_4309_86ed_f5d047e97c6f.slice/crio-c1ebe39d64d0744b71835cede8873d121137957ac01048158f96eb9093741ede WatchSource:0}: Error finding container c1ebe39d64d0744b71835cede8873d121137957ac01048158f96eb9093741ede: Status 404 returned error can't find the container with id c1ebe39d64d0744b71835cede8873d121137957ac01048158f96eb9093741ede
Jul 24 17:07:13.440238 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:13.440195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw" event={"ID":"d99834be-a9c5-4309-86ed-f5d047e97c6f","Type":"ContainerStarted","Data":"aa3470d84b1a59971b641c3ce1adcd882d01afe0cf10ce789449867db379c498"}
Jul 24 17:07:13.440467 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:13.440255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw" event={"ID":"d99834be-a9c5-4309-86ed-f5d047e97c6f","Type":"ContainerStarted","Data":"c1ebe39d64d0744b71835cede8873d121137957ac01048158f96eb9093741ede"}
Jul 24 17:07:13.440467 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:13.440294    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:07:13.462594 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:13.462536    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw" podStartSLOduration=1.46252131 podStartE2EDuration="1.46252131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:07:13.461652812 +0000 UTC m=+1440.221461823" watchObservedRunningTime="2026-07-24 17:07:13.46252131 +0000 UTC m=+1440.222330309"
Jul 24 17:07:24.447050 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:07:24.447017    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2r7cw"
Jul 24 17:09:56.187422 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.187389    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"]
Jul 24 17:09:56.190138 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.190122    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.193366 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.193343    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ec558e54-kserve-self-signed-certs\""
Jul 24 17:09:56.199504 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.199482    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"]
Jul 24 17:09:56.286975 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.286934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-dshm\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.287144 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.287011    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f8bh\" (UniqueName: \"kubernetes.io/projected/5a352116-9c39-4482-a1cb-eea89e28918e-kube-api-access-4f8bh\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.287144 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.287054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-model-cache\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.287144 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.287089    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-tmp-dir\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.287144 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.287139    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a352116-9c39-4482-a1cb-eea89e28918e-tls-certs\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.287299 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.287190    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-home\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388026 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.387988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f8bh\" (UniqueName: \"kubernetes.io/projected/5a352116-9c39-4482-a1cb-eea89e28918e-kube-api-access-4f8bh\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388220 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-model-cache\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388220 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-tmp-dir\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388220 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a352116-9c39-4482-a1cb-eea89e28918e-tls-certs\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388414 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-home\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388414 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-dshm\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388525 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388434    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-tmp-dir\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388525 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388508    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-model-cache\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.388614 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.388563    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-home\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.390581 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.390560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-dshm\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.390747 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.390730    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a352116-9c39-4482-a1cb-eea89e28918e-tls-certs\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.395759 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.395729    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f8bh\" (UniqueName: \"kubernetes.io/projected/5a352116-9c39-4482-a1cb-eea89e28918e-kube-api-access-4f8bh\") pod \"e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") " pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.499318 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.499232    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:09:56.639539 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:56.639512    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"]
Jul 24 17:09:56.642142 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:09:56.642102    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a352116_9c39_4482_a1cb_eea89e28918e.slice/crio-2f1abf038265ac09701a1a1c8cd5afe437bcda3e83c2a7dbaefd490f5d421048 WatchSource:0}: Error finding container 2f1abf038265ac09701a1a1c8cd5afe437bcda3e83c2a7dbaefd490f5d421048: Status 404 returned error can't find the container with id 2f1abf038265ac09701a1a1c8cd5afe437bcda3e83c2a7dbaefd490f5d421048
Jul 24 17:09:57.172702 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:57.172661    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn" event={"ID":"5a352116-9c39-4482-a1cb-eea89e28918e","Type":"ContainerStarted","Data":"7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb"}
Jul 24 17:09:57.172909 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:57.172712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn" event={"ID":"5a352116-9c39-4482-a1cb-eea89e28918e","Type":"ContainerStarted","Data":"2f1abf038265ac09701a1a1c8cd5afe437bcda3e83c2a7dbaefd490f5d421048"}
Jul 24 17:09:57.192644 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:09:57.192582    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn" podStartSLOduration=1.192565506 podStartE2EDuration="1.192565506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:09:57.190545002 +0000 UTC m=+1603.950353998" watchObservedRunningTime="2026-07-24 17:09:57.192565506 +0000 UTC m=+1603.952374504"
Jul 24 17:10:06.499856 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:06.499763    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:10:06.499856 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:06.499828    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:10:06.512185 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:06.512159    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:10:07.233126 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:07.233099    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:10:22.304466 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:22.304429    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"]
Jul 24 17:10:37.324060 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.323985    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn" podUID="5a352116-9c39-4482-a1cb-eea89e28918e" containerName="main" containerID="cri-o://7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb" gracePeriod=45
Jul 24 17:10:37.573620 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.573592    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:10:37.665104 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665024    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-dshm\") pod \"5a352116-9c39-4482-a1cb-eea89e28918e\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") "
Jul 24 17:10:37.665104 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665060    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-model-cache\") pod \"5a352116-9c39-4482-a1cb-eea89e28918e\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") "
Jul 24 17:10:37.665104 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665085    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-tmp-dir\") pod \"5a352116-9c39-4482-a1cb-eea89e28918e\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") "
Jul 24 17:10:37.665389 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665155    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f8bh\" (UniqueName: \"kubernetes.io/projected/5a352116-9c39-4482-a1cb-eea89e28918e-kube-api-access-4f8bh\") pod \"5a352116-9c39-4482-a1cb-eea89e28918e\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") "
Jul 24 17:10:37.665389 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665177    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-home\") pod \"5a352116-9c39-4482-a1cb-eea89e28918e\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") "
Jul 24 17:10:37.665389 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665215    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a352116-9c39-4482-a1cb-eea89e28918e-tls-certs\") pod \"5a352116-9c39-4482-a1cb-eea89e28918e\" (UID: \"5a352116-9c39-4482-a1cb-eea89e28918e\") "
Jul 24 17:10:37.665389 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665378    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-model-cache" (OuterVolumeSpecName: "model-cache") pod "5a352116-9c39-4482-a1cb-eea89e28918e" (UID: "5a352116-9c39-4482-a1cb-eea89e28918e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:10:37.665595 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665399    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5a352116-9c39-4482-a1cb-eea89e28918e" (UID: "5a352116-9c39-4482-a1cb-eea89e28918e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:10:37.665595 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665433    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-home" (OuterVolumeSpecName: "home") pod "5a352116-9c39-4482-a1cb-eea89e28918e" (UID: "5a352116-9c39-4482-a1cb-eea89e28918e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:10:37.665595 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665497    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:10:37.665595 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665511    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:10:37.665595 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.665523    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:10:37.667324 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.667293    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-dshm" (OuterVolumeSpecName: "dshm") pod "5a352116-9c39-4482-a1cb-eea89e28918e" (UID: "5a352116-9c39-4482-a1cb-eea89e28918e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:10:37.667324 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.667288    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a352116-9c39-4482-a1cb-eea89e28918e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5a352116-9c39-4482-a1cb-eea89e28918e" (UID: "5a352116-9c39-4482-a1cb-eea89e28918e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:10:37.667496 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.667367    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a352116-9c39-4482-a1cb-eea89e28918e-kube-api-access-4f8bh" (OuterVolumeSpecName: "kube-api-access-4f8bh") pod "5a352116-9c39-4482-a1cb-eea89e28918e" (UID: "5a352116-9c39-4482-a1cb-eea89e28918e"). InnerVolumeSpecName "kube-api-access-4f8bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:10:37.766974 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.766925    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f8bh\" (UniqueName: \"kubernetes.io/projected/5a352116-9c39-4482-a1cb-eea89e28918e-kube-api-access-4f8bh\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:10:37.766974 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.766970    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a352116-9c39-4482-a1cb-eea89e28918e-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:10:37.766974 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:37.766980    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a352116-9c39-4482-a1cb-eea89e28918e-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:10:38.360548 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.360508    2569 generic.go:356] "Generic (PLEG): container finished" podID="5a352116-9c39-4482-a1cb-eea89e28918e" containerID="7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb" exitCode=0
Jul 24 17:10:38.361022 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.360573    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"
Jul 24 17:10:38.361022 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.360577    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn" event={"ID":"5a352116-9c39-4482-a1cb-eea89e28918e","Type":"ContainerDied","Data":"7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb"}
Jul 24 17:10:38.361022 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.360682    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn" event={"ID":"5a352116-9c39-4482-a1cb-eea89e28918e","Type":"ContainerDied","Data":"2f1abf038265ac09701a1a1c8cd5afe437bcda3e83c2a7dbaefd490f5d421048"}
Jul 24 17:10:38.361022 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.360699    2569 scope.go:117] "RemoveContainer" containerID="7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb"
Jul 24 17:10:38.370314 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.370294    2569 scope.go:117] "RemoveContainer" containerID="7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb"
Jul 24 17:10:38.370609 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:10:38.370589    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb\": container with ID starting with 7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb not found: ID does not exist" containerID="7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb"
Jul 24 17:10:38.370672 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.370618    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb"} err="failed to get container status \"7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb\": rpc error: code = NotFound desc = could not find container \"7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb\": container with ID starting with 7b8dc6f808e13124e9e7180ff96e0c1fe57d66ad8728c36e3e9ad6fc4d79a6fb not found: ID does not exist"
Jul 24 17:10:38.382207 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.382180    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"]
Jul 24 17:10:38.384042 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:38.384019    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ec558e54-kserve-5b855864c9-n84pn"]
Jul 24 17:10:39.631340 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:10:39.631303    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a352116-9c39-4482-a1cb-eea89e28918e" path="/var/lib/kubelet/pods/5a352116-9c39-4482-a1cb-eea89e28918e/volumes"
Jul 24 17:14:34.604062 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.604026    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"]
Jul 24 17:14:34.604488 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.604393    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a352116-9c39-4482-a1cb-eea89e28918e" containerName="main"
Jul 24 17:14:34.604488 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.604404    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a352116-9c39-4482-a1cb-eea89e28918e" containerName="main"
Jul 24 17:14:34.604488 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.604468    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a352116-9c39-4482-a1cb-eea89e28918e" containerName="main"
Jul 24 17:14:34.607713 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.607697    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"
Jul 24 17:14:34.609947 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.609918    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:14:34.610056 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.610009    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:14:34.613615 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.613591    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"]
Jul 24 17:14:34.697295 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.697261    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e480aa-69ef-422b-8624-58f0c11212ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0275111b-2xnts\" (UID: \"91e480aa-69ef-422b-8624-58f0c11212ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"
Jul 24 17:14:34.798661 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.798620    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e480aa-69ef-422b-8624-58f0c11212ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0275111b-2xnts\" (UID: \"91e480aa-69ef-422b-8624-58f0c11212ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"
Jul 24 17:14:34.799257 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.799236    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e480aa-69ef-422b-8624-58f0c11212ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0275111b-2xnts\" (UID: \"91e480aa-69ef-422b-8624-58f0c11212ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"
Jul 24 17:14:34.919485 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:34.919409    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"
Jul 24 17:14:35.048903 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:35.048872    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"]
Jul 24 17:14:35.051786 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:14:35.051757    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e480aa_69ef_422b_8624_58f0c11212ad.slice/crio-70c6e39926e339cb198cf051bb5da57911272c4d802434757c70b109bbb1dc32 WatchSource:0}: Error finding container 70c6e39926e339cb198cf051bb5da57911272c4d802434757c70b109bbb1dc32: Status 404 returned error can't find the container with id 70c6e39926e339cb198cf051bb5da57911272c4d802434757c70b109bbb1dc32
Jul 24 17:14:35.053686 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:35.053666    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:14:35.415373 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:35.415321    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts" event={"ID":"91e480aa-69ef-422b-8624-58f0c11212ad","Type":"ContainerStarted","Data":"70c6e39926e339cb198cf051bb5da57911272c4d802434757c70b109bbb1dc32"}
Jul 24 17:14:37.425871 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:37.425834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts" event={"ID":"91e480aa-69ef-422b-8624-58f0c11212ad","Type":"ContainerStarted","Data":"f2eecc4ab178a02439b0e861f39e5589250764e9f26d4b6393496a576c609cc0"}
Jul 24 17:14:37.442314 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:37.442267    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts" podStartSLOduration=1.4358796489999999 podStartE2EDuration="3.442252109s" totalImagesPullingTime="2.00637246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:14:37.44125392 +0000 UTC m=+1884.201062920" watchObservedRunningTime="2026-07-24 17:14:37.442252109 +0000 UTC m=+1884.202061107"
Jul 24 17:14:38.431162 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:38.431123    2569 generic.go:356] "Generic (PLEG): container finished" podID="91e480aa-69ef-422b-8624-58f0c11212ad" containerID="f2eecc4ab178a02439b0e861f39e5589250764e9f26d4b6393496a576c609cc0" exitCode=0
Jul 24 17:14:38.431611 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:38.431205    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts" event={"ID":"91e480aa-69ef-422b-8624-58f0c11212ad","Type":"ContainerDied","Data":"f2eecc4ab178a02439b0e861f39e5589250764e9f26d4b6393496a576c609cc0"}
Jul 24 17:14:39.560867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:39.560838    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"
Jul 24 17:14:39.643783 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:39.643751    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e480aa-69ef-422b-8624-58f0c11212ad-maas-api-service-ca\") pod \"91e480aa-69ef-422b-8624-58f0c11212ad\" (UID: \"91e480aa-69ef-422b-8624-58f0c11212ad\") "
Jul 24 17:14:39.644090 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:39.644069    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91e480aa-69ef-422b-8624-58f0c11212ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91e480aa-69ef-422b-8624-58f0c11212ad" (UID: "91e480aa-69ef-422b-8624-58f0c11212ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:14:39.745357 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:39.745240    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e480aa-69ef-422b-8624-58f0c11212ad-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:14:40.440340 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:40.440304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts" event={"ID":"91e480aa-69ef-422b-8624-58f0c11212ad","Type":"ContainerDied","Data":"70c6e39926e339cb198cf051bb5da57911272c4d802434757c70b109bbb1dc32"}
Jul 24 17:14:40.440512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:40.440357    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"
Jul 24 17:14:40.440512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:40.440373    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70c6e39926e339cb198cf051bb5da57911272c4d802434757c70b109bbb1dc32"
Jul 24 17:14:49.680217 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:49.680177    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"]
Jul 24 17:14:49.684650 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:49.684622    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0275111b-2xnts"]
Jul 24 17:14:51.630084 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:14:51.630049    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91e480aa-69ef-422b-8624-58f0c11212ad" path="/var/lib/kubelet/pods/91e480aa-69ef-422b-8624-58f0c11212ad/volumes"
Jul 24 17:15:00.129439 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.129401    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl"]
Jul 24 17:15:00.129933 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.129916    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91e480aa-69ef-422b-8624-58f0c11212ad" containerName="revoke-keys"
Jul 24 17:15:00.129976 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.129937    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="91e480aa-69ef-422b-8624-58f0c11212ad" containerName="revoke-keys"
Jul 24 17:15:00.130028 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.130017    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="91e480aa-69ef-422b-8624-58f0c11212ad" containerName="revoke-keys"
Jul 24 17:15:00.133590 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.133572    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"]
Jul 24 17:15:00.133707 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.133691    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl"
Jul 24 17:15:00.136021 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.135996    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:15:00.136985 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.136971    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"
Jul 24 17:15:00.141005 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.140981    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl"]
Jul 24 17:15:00.145825 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.145804    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl"
Jul 24 17:15:00.153860 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.153837    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"
Jul 24 17:15:00.155016 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.154988    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"]
Jul 24 17:15:00.291832 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.291748    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl"]
Jul 24 17:15:00.295776 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:15:00.295742    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2ca7e0_c24b_4d79_9073_5baa506b5b12.slice/crio-70b246375e13c7f1b159f4bcf35bf1670f6b4bfd177e8d367d6b939447062a8c WatchSource:0}: Error finding container 70b246375e13c7f1b159f4bcf35bf1670f6b4bfd177e8d367d6b939447062a8c: Status 404 returned error can't find the container with id 70b246375e13c7f1b159f4bcf35bf1670f6b4bfd177e8d367d6b939447062a8c
Jul 24 17:15:00.319372 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.319345    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"]
Jul 24 17:15:00.333506 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:15:00.333475    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6322c50_c6ee_45d9_92f3_77537b1b726d.slice/crio-9b22ca216c8d31aee1ef180c4fa3d7c003eaab50f3d414d761cc7807da09ec7b WatchSource:0}: Error finding container 9b22ca216c8d31aee1ef180c4fa3d7c003eaab50f3d414d761cc7807da09ec7b: Status 404 returned error can't find the container with id 9b22ca216c8d31aee1ef180c4fa3d7c003eaab50f3d414d761cc7807da09ec7b
Jul 24 17:15:00.529648 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.529594    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7" event={"ID":"c6322c50-c6ee-45d9-92f3-77537b1b726d","Type":"ContainerStarted","Data":"9b22ca216c8d31aee1ef180c4fa3d7c003eaab50f3d414d761cc7807da09ec7b"}
Jul 24 17:15:00.531026 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:00.531003    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl" event={"ID":"9d2ca7e0-c24b-4d79-9073-5baa506b5b12","Type":"ContainerStarted","Data":"70b246375e13c7f1b159f4bcf35bf1670f6b4bfd177e8d367d6b939447062a8c"}
Jul 24 17:15:01.538233 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:01.538196    2569 generic.go:356] "Generic (PLEG): container finished" podID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerID="878f1a08d51f26d3556719ce4a77c172268baf09215eb6b8135102c35b22ce37" exitCode=7
Jul 24 17:15:01.538720 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:01.538272    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7" event={"ID":"c6322c50-c6ee-45d9-92f3-77537b1b726d","Type":"ContainerDied","Data":"878f1a08d51f26d3556719ce4a77c172268baf09215eb6b8135102c35b22ce37"}
Jul 24 17:15:01.538720 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:01.538473    2569 scope.go:117] "RemoveContainer" containerID="878f1a08d51f26d3556719ce4a77c172268baf09215eb6b8135102c35b22ce37"
Jul 24 17:15:01.539841 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:01.539815    2569 generic.go:356] "Generic (PLEG): container finished" podID="9d2ca7e0-c24b-4d79-9073-5baa506b5b12" containerID="fe0997979cb661f7f0c6b3465ac8fc35d83e31076ae2af92a1260058c5255d3e" exitCode=0
Jul 24 17:15:01.539945 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:01.539849    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl" event={"ID":"9d2ca7e0-c24b-4d79-9073-5baa506b5b12","Type":"ContainerDied","Data":"fe0997979cb661f7f0c6b3465ac8fc35d83e31076ae2af92a1260058c5255d3e"}
Jul 24 17:15:02.547310 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:02.547270    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7" event={"ID":"c6322c50-c6ee-45d9-92f3-77537b1b726d","Type":"ContainerStarted","Data":"dd9789dd72dfba576f2a79b04e0b72c10ff75172cb9a9ff77d2cf99b461f16d5"}
Jul 24 17:15:02.562805 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:02.562750    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7" podStartSLOduration=2.562733303 podStartE2EDuration="2.562733303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:02.561206656 +0000 UTC m=+1909.321015655" watchObservedRunningTime="2026-07-24 17:15:02.562733303 +0000 UTC m=+1909.322542301"
Jul 24 17:15:02.679013 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:02.678987    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl"
Jul 24 17:15:03.554178 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:03.554137    2569 generic.go:356] "Generic (PLEG): container finished" podID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerID="dd9789dd72dfba576f2a79b04e0b72c10ff75172cb9a9ff77d2cf99b461f16d5" exitCode=0
Jul 24 17:15:03.554625 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:03.554224    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7" event={"ID":"c6322c50-c6ee-45d9-92f3-77537b1b726d","Type":"ContainerDied","Data":"dd9789dd72dfba576f2a79b04e0b72c10ff75172cb9a9ff77d2cf99b461f16d5"}
Jul 24 17:15:03.554625 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:03.554287    2569 scope.go:117] "RemoveContainer" containerID="878f1a08d51f26d3556719ce4a77c172268baf09215eb6b8135102c35b22ce37"
Jul 24 17:15:03.555611 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:03.555514    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl" event={"ID":"9d2ca7e0-c24b-4d79-9073-5baa506b5b12","Type":"ContainerDied","Data":"70b246375e13c7f1b159f4bcf35bf1670f6b4bfd177e8d367d6b939447062a8c"}
Jul 24 17:15:03.555611 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:03.555540    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748555-kc6cl"
Jul 24 17:15:03.555611 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:03.555547    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70b246375e13c7f1b159f4bcf35bf1670f6b4bfd177e8d367d6b939447062a8c"
Jul 24 17:15:04.698701 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:04.698670    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"
Jul 24 17:15:05.567712 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:05.567679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7" event={"ID":"c6322c50-c6ee-45d9-92f3-77537b1b726d","Type":"ContainerDied","Data":"9b22ca216c8d31aee1ef180c4fa3d7c003eaab50f3d414d761cc7807da09ec7b"}
Jul 24 17:15:05.567891 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:05.567729    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b22ca216c8d31aee1ef180c4fa3d7c003eaab50f3d414d761cc7807da09ec7b"
Jul 24 17:15:05.567891 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:05.567689    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"
Jul 24 17:15:18.490207 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.490170    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"]
Jul 24 17:15:18.490614 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.490538    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerName="cleanup"
Jul 24 17:15:18.490614 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.490552    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerName="cleanup"
Jul 24 17:15:18.490614 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.490570    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2ca7e0-c24b-4d79-9073-5baa506b5b12" containerName="cleanup"
Jul 24 17:15:18.490614 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.490576    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2ca7e0-c24b-4d79-9073-5baa506b5b12" containerName="cleanup"
Jul 24 17:15:18.490744 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.490640    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerName="cleanup"
Jul 24 17:15:18.490744 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.490651    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2ca7e0-c24b-4d79-9073-5baa506b5b12" containerName="cleanup"
Jul 24 17:15:18.493675 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.493659    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"
Jul 24 17:15:18.496031 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.496009    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:15:18.496126 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.496082    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:15:18.500632 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.500597    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"]
Jul 24 17:15:18.585265 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.585226    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e54b0c3-8c53-41ac-9533-1844f2e4f98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm\" (UID: \"6e54b0c3-8c53-41ac-9533-1844f2e4f98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"
Jul 24 17:15:18.686594 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.686555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e54b0c3-8c53-41ac-9533-1844f2e4f98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm\" (UID: \"6e54b0c3-8c53-41ac-9533-1844f2e4f98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"
Jul 24 17:15:18.687179 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.687161    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e54b0c3-8c53-41ac-9533-1844f2e4f98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm\" (UID: \"6e54b0c3-8c53-41ac-9533-1844f2e4f98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"
Jul 24 17:15:18.804960 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:18.804873    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"
Jul 24 17:15:19.137050 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:19.137026    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"]
Jul 24 17:15:19.139070 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:15:19.139046    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e54b0c3_8c53_41ac_9533_1844f2e4f98e.slice/crio-844e289255cfdee667eb2c55d8e665b01b2d76f4941868c6018476697826db35 WatchSource:0}: Error finding container 844e289255cfdee667eb2c55d8e665b01b2d76f4941868c6018476697826db35: Status 404 returned error can't find the container with id 844e289255cfdee667eb2c55d8e665b01b2d76f4941868c6018476697826db35
Jul 24 17:15:19.633444 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:19.633412    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm" event={"ID":"6e54b0c3-8c53-41ac-9533-1844f2e4f98e","Type":"ContainerStarted","Data":"5728262e21698e9b4f1c3ab6aba4e82d14b45023f0be9127e6905d3ef9b18db7"}
Jul 24 17:15:19.633778 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:19.633454    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm" event={"ID":"6e54b0c3-8c53-41ac-9533-1844f2e4f98e","Type":"ContainerStarted","Data":"844e289255cfdee667eb2c55d8e665b01b2d76f4941868c6018476697826db35"}
Jul 24 17:15:19.649641 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:19.649596    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm" podStartSLOduration=1.6495800790000001 podStartE2EDuration="1.649580079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:19.648112482 +0000 UTC m=+1926.407921481" watchObservedRunningTime="2026-07-24 17:15:19.649580079 +0000 UTC m=+1926.409389086"
Jul 24 17:15:20.639453 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:20.639414    2569 generic.go:356] "Generic (PLEG): container finished" podID="6e54b0c3-8c53-41ac-9533-1844f2e4f98e" containerID="5728262e21698e9b4f1c3ab6aba4e82d14b45023f0be9127e6905d3ef9b18db7" exitCode=0
Jul 24 17:15:20.639915 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:20.639500    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm" event={"ID":"6e54b0c3-8c53-41ac-9533-1844f2e4f98e","Type":"ContainerDied","Data":"5728262e21698e9b4f1c3ab6aba4e82d14b45023f0be9127e6905d3ef9b18db7"}
Jul 24 17:15:21.770701 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:21.770675    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"
Jul 24 17:15:21.813384 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:21.813353    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e54b0c3-8c53-41ac-9533-1844f2e4f98e-maas-api-service-ca\") pod \"6e54b0c3-8c53-41ac-9533-1844f2e4f98e\" (UID: \"6e54b0c3-8c53-41ac-9533-1844f2e4f98e\") "
Jul 24 17:15:21.813686 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:21.813663    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e54b0c3-8c53-41ac-9533-1844f2e4f98e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e54b0c3-8c53-41ac-9533-1844f2e4f98e" (UID: "6e54b0c3-8c53-41ac-9533-1844f2e4f98e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:15:21.914090 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:21.913999    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e54b0c3-8c53-41ac-9533-1844f2e4f98e-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:15:22.649445 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:22.649408    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"
Jul 24 17:15:22.649445 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:22.649418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm" event={"ID":"6e54b0c3-8c53-41ac-9533-1844f2e4f98e","Type":"ContainerDied","Data":"844e289255cfdee667eb2c55d8e665b01b2d76f4941868c6018476697826db35"}
Jul 24 17:15:22.649656 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:22.649475    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="844e289255cfdee667eb2c55d8e665b01b2d76f4941868c6018476697826db35"
Jul 24 17:15:29.063597 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:29.063550    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"]
Jul 24 17:15:29.065501 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:29.065482    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-9651e0f7-29748555-6srf7"]
Jul 24 17:15:29.629797 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:29.629762    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6322c50-c6ee-45d9-92f3-77537b1b726d" path="/var/lib/kubelet/pods/c6322c50-c6ee-45d9-92f3-77537b1b726d/volumes"
Jul 24 17:15:33.570583 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:33.570545    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"]
Jul 24 17:15:33.573536 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:33.573510    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-9651e0f7-95snm"]
Jul 24 17:15:33.630431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:15:33.630389    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e54b0c3-8c53-41ac-9533-1844f2e4f98e" path="/var/lib/kubelet/pods/6e54b0c3-8c53-41ac-9533-1844f2e4f98e/volumes"
Jul 24 17:16:02.387941 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.387893    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"]
Jul 24 17:16:02.390533 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.388250    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e54b0c3-8c53-41ac-9533-1844f2e4f98e" containerName="revoke-keys"
Jul 24 17:16:02.390533 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.388261    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e54b0c3-8c53-41ac-9533-1844f2e4f98e" containerName="revoke-keys"
Jul 24 17:16:02.390533 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.388269    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerName="cleanup"
Jul 24 17:16:02.390533 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.388275    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerName="cleanup"
Jul 24 17:16:02.390533 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.388355    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6322c50-c6ee-45d9-92f3-77537b1b726d" containerName="cleanup"
Jul 24 17:16:02.390533 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.388363    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e54b0c3-8c53-41ac-9533-1844f2e4f98e" containerName="revoke-keys"
Jul 24 17:16:02.391573 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.391556    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"
Jul 24 17:16:02.394105 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.394081    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:16:02.394223 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.394087    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:16:02.397965 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.397939    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"]
Jul 24 17:16:02.475720 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.475681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl\" (UID: \"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"
Jul 24 17:16:02.576686 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.576647    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl\" (UID: \"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"
Jul 24 17:16:02.577251 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.577232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl\" (UID: \"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"
Jul 24 17:16:02.703583 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.703482    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"
Jul 24 17:16:02.835366 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:02.835321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"]
Jul 24 17:16:02.837645 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:16:02.837620    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe10eee1_b4c7_4f0b_aaab_84a7b7c959ee.slice/crio-bbecdf01e06179ff09c616b9eb00cef691f8331413b2d6914a46752872e76fe2 WatchSource:0}: Error finding container bbecdf01e06179ff09c616b9eb00cef691f8331413b2d6914a46752872e76fe2: Status 404 returned error can't find the container with id bbecdf01e06179ff09c616b9eb00cef691f8331413b2d6914a46752872e76fe2
Jul 24 17:16:03.842905 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:03.842867    2569 generic.go:356] "Generic (PLEG): container finished" podID="fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee" containerID="0a3b93ecaf25c77fc9fbfc770abeddbd93b0881fe76e7918869c2512adafacae" exitCode=0
Jul 24 17:16:03.843276 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:03.842915    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl" event={"ID":"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee","Type":"ContainerDied","Data":"0a3b93ecaf25c77fc9fbfc770abeddbd93b0881fe76e7918869c2512adafacae"}
Jul 24 17:16:03.843276 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:03.842972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl" event={"ID":"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee","Type":"ContainerStarted","Data":"bbecdf01e06179ff09c616b9eb00cef691f8331413b2d6914a46752872e76fe2"}
Jul 24 17:16:04.977243 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:04.977222    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"
Jul 24 17:16:04.993097 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:04.993068    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee-maas-api-service-ca\") pod \"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee\" (UID: \"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee\") "
Jul 24 17:16:04.993602 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:04.993564    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee" (UID: "fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:16:05.094403 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:05.094364    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:16:05.853293 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:05.853229    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"
Jul 24 17:16:05.853293 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:05.853273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl" event={"ID":"fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee","Type":"ContainerDied","Data":"bbecdf01e06179ff09c616b9eb00cef691f8331413b2d6914a46752872e76fe2"}
Jul 24 17:16:05.853566 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:05.853343    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbecdf01e06179ff09c616b9eb00cef691f8331413b2d6914a46752872e76fe2"
Jul 24 17:16:17.464873 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:17.464831    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"]
Jul 24 17:16:17.467848 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:17.467803    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9300b6a3-5tdkl"]
Jul 24 17:16:17.630223 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:17.630191    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee" path="/var/lib/kubelet/pods/fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee/volumes"
Jul 24 17:16:46.772083 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.772053    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"]
Jul 24 17:16:46.772555 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.772399    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee" containerName="revoke-keys"
Jul 24 17:16:46.772555 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.772412    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee" containerName="revoke-keys"
Jul 24 17:16:46.772555 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.772480    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe10eee1-b4c7-4f0b-aaab-84a7b7c959ee" containerName="revoke-keys"
Jul 24 17:16:46.775649 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.775619    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"
Jul 24 17:16:46.778215 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.778192    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:16:46.778345 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.778244    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:16:46.781842 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.781819    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"]
Jul 24 17:16:46.864904 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.864873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/897b268f-2e24-421e-9439-5fb9a83d977a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57de5652-j9lhd\" (UID: \"897b268f-2e24-421e-9439-5fb9a83d977a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"
Jul 24 17:16:46.966159 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.966117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/897b268f-2e24-421e-9439-5fb9a83d977a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57de5652-j9lhd\" (UID: \"897b268f-2e24-421e-9439-5fb9a83d977a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"
Jul 24 17:16:46.966852 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:46.966830    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/897b268f-2e24-421e-9439-5fb9a83d977a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57de5652-j9lhd\" (UID: \"897b268f-2e24-421e-9439-5fb9a83d977a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"
Jul 24 17:16:47.086557 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:47.086528    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"
Jul 24 17:16:47.218504 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:47.218465    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"]
Jul 24 17:16:47.222153 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:16:47.222130    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897b268f_2e24_421e_9439_5fb9a83d977a.slice/crio-836ecc409d66af1ea27db6bbc77d1a550d1ac32cbe76738736780aa84c578b4c WatchSource:0}: Error finding container 836ecc409d66af1ea27db6bbc77d1a550d1ac32cbe76738736780aa84c578b4c: Status 404 returned error can't find the container with id 836ecc409d66af1ea27db6bbc77d1a550d1ac32cbe76738736780aa84c578b4c
Jul 24 17:16:48.056531 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:48.056491    2569 generic.go:356] "Generic (PLEG): container finished" podID="897b268f-2e24-421e-9439-5fb9a83d977a" containerID="230f009acec8d23a7c5246f9020b30c0043f67ee5822c2a2c2b689fe4a4ad06d" exitCode=0
Jul 24 17:16:48.056902 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:48.056572    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd" event={"ID":"897b268f-2e24-421e-9439-5fb9a83d977a","Type":"ContainerDied","Data":"230f009acec8d23a7c5246f9020b30c0043f67ee5822c2a2c2b689fe4a4ad06d"}
Jul 24 17:16:48.056902 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:48.056665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd" event={"ID":"897b268f-2e24-421e-9439-5fb9a83d977a","Type":"ContainerStarted","Data":"836ecc409d66af1ea27db6bbc77d1a550d1ac32cbe76738736780aa84c578b4c"}
Jul 24 17:16:49.188997 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:49.188973    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"
Jul 24 17:16:49.286437 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:49.286402    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/897b268f-2e24-421e-9439-5fb9a83d977a-maas-api-service-ca\") pod \"897b268f-2e24-421e-9439-5fb9a83d977a\" (UID: \"897b268f-2e24-421e-9439-5fb9a83d977a\") "
Jul 24 17:16:49.286788 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:49.286765    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/897b268f-2e24-421e-9439-5fb9a83d977a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "897b268f-2e24-421e-9439-5fb9a83d977a" (UID: "897b268f-2e24-421e-9439-5fb9a83d977a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:16:49.387303 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:49.387272    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/897b268f-2e24-421e-9439-5fb9a83d977a-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:16:50.066350 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:50.066298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd" event={"ID":"897b268f-2e24-421e-9439-5fb9a83d977a","Type":"ContainerDied","Data":"836ecc409d66af1ea27db6bbc77d1a550d1ac32cbe76738736780aa84c578b4c"}
Jul 24 17:16:50.066521 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:50.066367    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="836ecc409d66af1ea27db6bbc77d1a550d1ac32cbe76738736780aa84c578b4c"
Jul 24 17:16:50.066521 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:16:50.066315    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"
Jul 24 17:17:01.846017 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:17:01.845979    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"]
Jul 24 17:17:01.848941 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:17:01.848911    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57de5652-j9lhd"]
Jul 24 17:17:03.630191 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:17:03.630150    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897b268f-2e24-421e-9439-5fb9a83d977a" path="/var/lib/kubelet/pods/897b268f-2e24-421e-9439-5fb9a83d977a/volumes"
Jul 24 17:18:55.039675 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.039641    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"]
Jul 24 17:18:55.040089 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.039978    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897b268f-2e24-421e-9439-5fb9a83d977a" containerName="revoke-keys"
Jul 24 17:18:55.040089 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.039990    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="897b268f-2e24-421e-9439-5fb9a83d977a" containerName="revoke-keys"
Jul 24 17:18:55.040089 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.040055    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="897b268f-2e24-421e-9439-5fb9a83d977a" containerName="revoke-keys"
Jul 24 17:18:55.043127 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.043111    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"
Jul 24 17:18:55.045546 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.045525    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:18:55.045546 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.045540    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:18:55.050875 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.050849    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"]
Jul 24 17:18:55.114245 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.114213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86dcfe23-344c-477c-b63f-661368908db7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h\" (UID: \"86dcfe23-344c-477c-b63f-661368908db7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"
Jul 24 17:18:55.214701 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.214668    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86dcfe23-344c-477c-b63f-661368908db7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h\" (UID: \"86dcfe23-344c-477c-b63f-661368908db7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"
Jul 24 17:18:55.215321 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.215301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86dcfe23-344c-477c-b63f-661368908db7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h\" (UID: \"86dcfe23-344c-477c-b63f-661368908db7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"
Jul 24 17:18:55.355186 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.355151    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"
Jul 24 17:18:55.483679 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.483654    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"]
Jul 24 17:18:55.486994 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:18:55.486964    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86dcfe23_344c_477c_b63f_661368908db7.slice/crio-f2d891db57ad7d7d46839537764041ef1e832e224f6a9c7d38374a31818aee1c WatchSource:0}: Error finding container f2d891db57ad7d7d46839537764041ef1e832e224f6a9c7d38374a31818aee1c: Status 404 returned error can't find the container with id f2d891db57ad7d7d46839537764041ef1e832e224f6a9c7d38374a31818aee1c
Jul 24 17:18:55.658748 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:55.658661    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h" event={"ID":"86dcfe23-344c-477c-b63f-661368908db7","Type":"ContainerStarted","Data":"f2d891db57ad7d7d46839537764041ef1e832e224f6a9c7d38374a31818aee1c"}
Jul 24 17:18:56.665115 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:56.665085    2569 generic.go:356] "Generic (PLEG): container finished" podID="86dcfe23-344c-477c-b63f-661368908db7" containerID="7fe1d2493598f949729fec817da370756b70a48f4ef2e4621e6eac4f493ba785" exitCode=0
Jul 24 17:18:56.665515 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:56.665156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h" event={"ID":"86dcfe23-344c-477c-b63f-661368908db7","Type":"ContainerDied","Data":"7fe1d2493598f949729fec817da370756b70a48f4ef2e4621e6eac4f493ba785"}
Jul 24 17:18:57.798260 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:57.798233    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"
Jul 24 17:18:57.940010 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:57.939922    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86dcfe23-344c-477c-b63f-661368908db7-maas-api-service-ca\") pod \"86dcfe23-344c-477c-b63f-661368908db7\" (UID: \"86dcfe23-344c-477c-b63f-661368908db7\") "
Jul 24 17:18:57.940283 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:57.940261    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86dcfe23-344c-477c-b63f-661368908db7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86dcfe23-344c-477c-b63f-661368908db7" (UID: "86dcfe23-344c-477c-b63f-661368908db7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:18:58.041545 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:58.041511    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86dcfe23-344c-477c-b63f-661368908db7-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:18:58.674779 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:58.674745    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h" event={"ID":"86dcfe23-344c-477c-b63f-661368908db7","Type":"ContainerDied","Data":"f2d891db57ad7d7d46839537764041ef1e832e224f6a9c7d38374a31818aee1c"}
Jul 24 17:18:58.674925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:58.674795    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2d891db57ad7d7d46839537764041ef1e832e224f6a9c7d38374a31818aee1c"
Jul 24 17:18:58.674925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:18:58.674761    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"
Jul 24 17:19:10.118244 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:19:10.118162    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"]
Jul 24 17:19:10.121488 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:19:10.121465    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e30ccfde-b2m2h"]
Jul 24 17:19:11.630465 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:19:11.630430    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86dcfe23-344c-477c-b63f-661368908db7" path="/var/lib/kubelet/pods/86dcfe23-344c-477c-b63f-661368908db7/volumes"
Jul 24 17:20:17.681183 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.681148    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"]
Jul 24 17:20:17.681637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.681500    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86dcfe23-344c-477c-b63f-661368908db7" containerName="revoke-keys"
Jul 24 17:20:17.681637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.681511    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="86dcfe23-344c-477c-b63f-661368908db7" containerName="revoke-keys"
Jul 24 17:20:17.681637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.681582    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="86dcfe23-344c-477c-b63f-661368908db7" containerName="revoke-keys"
Jul 24 17:20:17.684552 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.684538    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"
Jul 24 17:20:17.686798 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.686778    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:20:17.686798 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.686789    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:20:17.691050 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.691025    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"]
Jul 24 17:20:17.753985 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.753948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/234bb435-f7e7-4d2a-968d-fafb2cf348f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq\" (UID: \"234bb435-f7e7-4d2a-968d-fafb2cf348f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"
Jul 24 17:20:17.855032 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.854996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/234bb435-f7e7-4d2a-968d-fafb2cf348f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq\" (UID: \"234bb435-f7e7-4d2a-968d-fafb2cf348f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"
Jul 24 17:20:17.855673 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.855654    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/234bb435-f7e7-4d2a-968d-fafb2cf348f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq\" (UID: \"234bb435-f7e7-4d2a-968d-fafb2cf348f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"
Jul 24 17:20:17.995644 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:17.995566    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"
Jul 24 17:20:18.128993 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:18.128959    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"]
Jul 24 17:20:18.132940 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:20:18.132904    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234bb435_f7e7_4d2a_968d_fafb2cf348f8.slice/crio-e80571625693c1fc161a6828cfe785fe1f845bcc496f200ab6c8734f0ee4e505 WatchSource:0}: Error finding container e80571625693c1fc161a6828cfe785fe1f845bcc496f200ab6c8734f0ee4e505: Status 404 returned error can't find the container with id e80571625693c1fc161a6828cfe785fe1f845bcc496f200ab6c8734f0ee4e505
Jul 24 17:20:19.067613 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:19.067587    2569 generic.go:356] "Generic (PLEG): container finished" podID="234bb435-f7e7-4d2a-968d-fafb2cf348f8" containerID="5728d6311861c296b583442777b7d70afa342cb3d949bc0e51401d68df2c882e" exitCode=0
Jul 24 17:20:19.068056 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:19.067653    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq" event={"ID":"234bb435-f7e7-4d2a-968d-fafb2cf348f8","Type":"ContainerDied","Data":"5728d6311861c296b583442777b7d70afa342cb3d949bc0e51401d68df2c882e"}
Jul 24 17:20:19.068056 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:19.067690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq" event={"ID":"234bb435-f7e7-4d2a-968d-fafb2cf348f8","Type":"ContainerStarted","Data":"e80571625693c1fc161a6828cfe785fe1f845bcc496f200ab6c8734f0ee4e505"}
Jul 24 17:20:20.199789 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:20.199758    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"
Jul 24 17:20:20.275804 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:20.275764    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/234bb435-f7e7-4d2a-968d-fafb2cf348f8-maas-api-service-ca\") pod \"234bb435-f7e7-4d2a-968d-fafb2cf348f8\" (UID: \"234bb435-f7e7-4d2a-968d-fafb2cf348f8\") "
Jul 24 17:20:20.276136 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:20.276115    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/234bb435-f7e7-4d2a-968d-fafb2cf348f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "234bb435-f7e7-4d2a-968d-fafb2cf348f8" (UID: "234bb435-f7e7-4d2a-968d-fafb2cf348f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:20:20.376553 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:20.376471    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/234bb435-f7e7-4d2a-968d-fafb2cf348f8-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:20:21.078267 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:21.078224    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq" event={"ID":"234bb435-f7e7-4d2a-968d-fafb2cf348f8","Type":"ContainerDied","Data":"e80571625693c1fc161a6828cfe785fe1f845bcc496f200ab6c8734f0ee4e505"}
Jul 24 17:20:21.078267 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:21.078246    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"
Jul 24 17:20:21.078512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:21.078288    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e80571625693c1fc161a6828cfe785fe1f845bcc496f200ab6c8734f0ee4e505"
Jul 24 17:20:37.765290 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:37.765209    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"]
Jul 24 17:20:37.769055 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:37.769025    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e27a909-fs2sq"]
Jul 24 17:20:39.635761 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:20:39.635717    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="234bb435-f7e7-4d2a-968d-fafb2cf348f8" path="/var/lib/kubelet/pods/234bb435-f7e7-4d2a-968d-fafb2cf348f8/volumes"
Jul 24 17:21:13.790154 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:21:13.790124    2569 scope.go:117] "RemoveContainer" containerID="dd9789dd72dfba576f2a79b04e0b72c10ff75172cb9a9ff77d2cf99b461f16d5"
Jul 24 17:21:13.798118 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:21:13.798100    2569 scope.go:117] "RemoveContainer" containerID="f2eecc4ab178a02439b0e861f39e5589250764e9f26d4b6393496a576c609cc0"
Jul 24 17:22:13.822354 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:13.822261    2569 scope.go:117] "RemoveContainer" containerID="0a3b93ecaf25c77fc9fbfc770abeddbd93b0881fe76e7918869c2512adafacae"
Jul 24 17:22:13.830225 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:13.830202    2569 scope.go:117] "RemoveContainer" containerID="5728262e21698e9b4f1c3ab6aba4e82d14b45023f0be9127e6905d3ef9b18db7"
Jul 24 17:22:49.977170 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.977132    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"]
Jul 24 17:22:49.977598 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.977487    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234bb435-f7e7-4d2a-968d-fafb2cf348f8" containerName="revoke-keys"
Jul 24 17:22:49.977598 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.977498    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="234bb435-f7e7-4d2a-968d-fafb2cf348f8" containerName="revoke-keys"
Jul 24 17:22:49.977598 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.977564    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="234bb435-f7e7-4d2a-968d-fafb2cf348f8" containerName="revoke-keys"
Jul 24 17:22:49.980658 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.980640    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:49.983229 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.983209    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0a7def\"/\"openshift-service-ca.crt\""
Jul 24 17:22:49.984245 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.984226    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0a7def\"/\"default-dockercfg-lwqd6\""
Jul 24 17:22:49.984294 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.984280    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0a7def\"/\"test-model-0a7def-kserve-self-signed-certs\""
Jul 24 17:22:49.984398 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.984383    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0a7def\"/\"kube-root-ca.crt\""
Jul 24 17:22:49.989886 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:49.989860    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"]
Jul 24 17:22:50.102418 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.102376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95013af-8656-453e-baa5-4c7b9eeccb15-tls-certs\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.102580 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.102444    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-dshm\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.102580 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.102475    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-model-cache\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.102580 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.102511    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-home\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.102580 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.102549    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs8n2\" (UniqueName: \"kubernetes.io/projected/e95013af-8656-453e-baa5-4c7b9eeccb15-kube-api-access-xs8n2\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.102747 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.102586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-tmp-dir\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.203871 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.203777    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-dshm\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.203871 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.203815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-model-cache\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.203871 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.203841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-home\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.204151 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.203873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs8n2\" (UniqueName: \"kubernetes.io/projected/e95013af-8656-453e-baa5-4c7b9eeccb15-kube-api-access-xs8n2\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.204151 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.203905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-tmp-dir\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.204151 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.203925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95013af-8656-453e-baa5-4c7b9eeccb15-tls-certs\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.204349 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.204304    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-tmp-dir\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.204431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.204404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-home\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.204482 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.204427    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-model-cache\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.206073 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.206053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-dshm\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.206317 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.206301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95013af-8656-453e-baa5-4c7b9eeccb15-tls-certs\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.212065 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.212044    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs8n2\" (UniqueName: \"kubernetes.io/projected/e95013af-8656-453e-baa5-4c7b9eeccb15-kube-api-access-xs8n2\") pod \"test-model-0a7def-kserve-7585d48ff9-49bl8\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") " pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.289872 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.289832    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:22:50.418111 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.418083    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"]
Jul 24 17:22:50.420992 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:22:50.420958    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode95013af_8656_453e_baa5_4c7b9eeccb15.slice/crio-50006b5770f8bda41093575c74eeed4046c90ae7a8590a1ef70c65c92af3bcb0 WatchSource:0}: Error finding container 50006b5770f8bda41093575c74eeed4046c90ae7a8590a1ef70c65c92af3bcb0: Status 404 returned error can't find the container with id 50006b5770f8bda41093575c74eeed4046c90ae7a8590a1ef70c65c92af3bcb0
Jul 24 17:22:50.783739 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.783650    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8" event={"ID":"e95013af-8656-453e-baa5-4c7b9eeccb15","Type":"ContainerStarted","Data":"0b45160e2f2e14176f15d26315aca3cb39d7289327e824713ef43806bcad38df"}
Jul 24 17:22:50.783739 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.783690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8" event={"ID":"e95013af-8656-453e-baa5-4c7b9eeccb15","Type":"ContainerStarted","Data":"50006b5770f8bda41093575c74eeed4046c90ae7a8590a1ef70c65c92af3bcb0"}
Jul 24 17:22:50.802979 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:22:50.802934    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8" podStartSLOduration=1.802919378 podStartE2EDuration="1.802919378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:22:50.800066707 +0000 UTC m=+2377.559875705" watchObservedRunningTime="2026-07-24 17:22:50.802919378 +0000 UTC m=+2377.562728377"
Jul 24 17:23:00.290541 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:23:00.290493    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:23:00.290541 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:23:00.290549    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:23:00.303102 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:23:00.303076    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:23:00.840681 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:23:00.840654    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:23:13.850648 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:23:13.850619    2569 scope.go:117] "RemoveContainer" containerID="230f009acec8d23a7c5246f9020b30c0043f67ee5822c2a2c2b689fe4a4ad06d"
Jul 24 17:24:00.962774 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:00.962728    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"]
Jul 24 17:24:02.035966 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.035930    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"]
Jul 24 17:24:02.039562 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.039544    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"
Jul 24 17:24:02.041996 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.041975    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:24:02.042100 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.041978    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:24:02.047994 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.047961    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"]
Jul 24 17:24:02.151904 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.151872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17165019-aa97-48dd-a8e1-24e88a3142b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk\" (UID: \"17165019-aa97-48dd-a8e1-24e88a3142b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"
Jul 24 17:24:02.252580 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.252541    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17165019-aa97-48dd-a8e1-24e88a3142b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk\" (UID: \"17165019-aa97-48dd-a8e1-24e88a3142b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"
Jul 24 17:24:02.253154 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.253131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17165019-aa97-48dd-a8e1-24e88a3142b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk\" (UID: \"17165019-aa97-48dd-a8e1-24e88a3142b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"
Jul 24 17:24:02.351400 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.351365    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"
Jul 24 17:24:02.483816 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:02.483783    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"]
Jul 24 17:24:02.486511 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:24:02.486483    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17165019_aa97_48dd_a8e1_24e88a3142b6.slice/crio-5082cfef96f60c8612b8290968f625bef0659ccf76b2a3f1fd28549d2c25ffc6 WatchSource:0}: Error finding container 5082cfef96f60c8612b8290968f625bef0659ccf76b2a3f1fd28549d2c25ffc6: Status 404 returned error can't find the container with id 5082cfef96f60c8612b8290968f625bef0659ccf76b2a3f1fd28549d2c25ffc6
Jul 24 17:24:03.121079 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:03.121042    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk" event={"ID":"17165019-aa97-48dd-a8e1-24e88a3142b6","Type":"ContainerStarted","Data":"d07ca2747252a45a17ed5e56908d341d66c73595ad4643762383b70f3c3101a1"}
Jul 24 17:24:03.121572 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:03.121106    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk" event={"ID":"17165019-aa97-48dd-a8e1-24e88a3142b6","Type":"ContainerStarted","Data":"5082cfef96f60c8612b8290968f625bef0659ccf76b2a3f1fd28549d2c25ffc6"}
Jul 24 17:24:03.137553 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:03.137503    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk" podStartSLOduration=1.137488315 podStartE2EDuration="1.137488315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:24:03.135417167 +0000 UTC m=+2449.895226167" watchObservedRunningTime="2026-07-24 17:24:03.137488315 +0000 UTC m=+2449.897297314"
Jul 24 17:24:04.128755 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:04.128716    2569 generic.go:356] "Generic (PLEG): container finished" podID="17165019-aa97-48dd-a8e1-24e88a3142b6" containerID="d07ca2747252a45a17ed5e56908d341d66c73595ad4643762383b70f3c3101a1" exitCode=0
Jul 24 17:24:04.129148 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:04.128804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk" event={"ID":"17165019-aa97-48dd-a8e1-24e88a3142b6","Type":"ContainerDied","Data":"d07ca2747252a45a17ed5e56908d341d66c73595ad4643762383b70f3c3101a1"}
Jul 24 17:24:05.267946 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:05.267920    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"
Jul 24 17:24:05.381915 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:05.381885    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17165019-aa97-48dd-a8e1-24e88a3142b6-maas-api-service-ca\") pod \"17165019-aa97-48dd-a8e1-24e88a3142b6\" (UID: \"17165019-aa97-48dd-a8e1-24e88a3142b6\") "
Jul 24 17:24:05.382261 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:05.382233    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17165019-aa97-48dd-a8e1-24e88a3142b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17165019-aa97-48dd-a8e1-24e88a3142b6" (UID: "17165019-aa97-48dd-a8e1-24e88a3142b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:24:05.482576 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:05.482483    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17165019-aa97-48dd-a8e1-24e88a3142b6-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:06.139025 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:06.138991    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"
Jul 24 17:24:06.139181 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:06.139024    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk" event={"ID":"17165019-aa97-48dd-a8e1-24e88a3142b6","Type":"ContainerDied","Data":"5082cfef96f60c8612b8290968f625bef0659ccf76b2a3f1fd28549d2c25ffc6"}
Jul 24 17:24:06.139181 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:06.139072    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5082cfef96f60c8612b8290968f625bef0659ccf76b2a3f1fd28549d2c25ffc6"
Jul 24 17:24:15.982029 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:15.981972    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8" podUID="e95013af-8656-453e-baa5-4c7b9eeccb15" containerName="main" containerID="cri-o://0b45160e2f2e14176f15d26315aca3cb39d7289327e824713ef43806bcad38df" gracePeriod=45
Jul 24 17:24:16.186599 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.186569    2569 generic.go:356] "Generic (PLEG): container finished" podID="e95013af-8656-453e-baa5-4c7b9eeccb15" containerID="0b45160e2f2e14176f15d26315aca3cb39d7289327e824713ef43806bcad38df" exitCode=0
Jul 24 17:24:16.186753 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.186641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8" event={"ID":"e95013af-8656-453e-baa5-4c7b9eeccb15","Type":"ContainerDied","Data":"0b45160e2f2e14176f15d26315aca3cb39d7289327e824713ef43806bcad38df"}
Jul 24 17:24:16.229382 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.229354    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:24:16.382786 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.382743    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-tmp-dir\") pod \"e95013af-8656-453e-baa5-4c7b9eeccb15\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") "
Jul 24 17:24:16.382786 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.382791    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95013af-8656-453e-baa5-4c7b9eeccb15-tls-certs\") pod \"e95013af-8656-453e-baa5-4c7b9eeccb15\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") "
Jul 24 17:24:16.383029 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.382809    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-model-cache\") pod \"e95013af-8656-453e-baa5-4c7b9eeccb15\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") "
Jul 24 17:24:16.383029 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.382846    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-dshm\") pod \"e95013af-8656-453e-baa5-4c7b9eeccb15\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") "
Jul 24 17:24:16.383029 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.382870    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-home\") pod \"e95013af-8656-453e-baa5-4c7b9eeccb15\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") "
Jul 24 17:24:16.383029 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.382917    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs8n2\" (UniqueName: \"kubernetes.io/projected/e95013af-8656-453e-baa5-4c7b9eeccb15-kube-api-access-xs8n2\") pod \"e95013af-8656-453e-baa5-4c7b9eeccb15\" (UID: \"e95013af-8656-453e-baa5-4c7b9eeccb15\") "
Jul 24 17:24:16.383236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.383097    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e95013af-8656-453e-baa5-4c7b9eeccb15" (UID: "e95013af-8656-453e-baa5-4c7b9eeccb15"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:24:16.383236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.383151    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-model-cache" (OuterVolumeSpecName: "model-cache") pod "e95013af-8656-453e-baa5-4c7b9eeccb15" (UID: "e95013af-8656-453e-baa5-4c7b9eeccb15"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:24:16.383236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.383188    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-home" (OuterVolumeSpecName: "home") pod "e95013af-8656-453e-baa5-4c7b9eeccb15" (UID: "e95013af-8656-453e-baa5-4c7b9eeccb15"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:24:16.383236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.383212    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:16.385102 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.385075    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e95013af-8656-453e-baa5-4c7b9eeccb15-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e95013af-8656-453e-baa5-4c7b9eeccb15" (UID: "e95013af-8656-453e-baa5-4c7b9eeccb15"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:24:16.385102 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.385087    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-dshm" (OuterVolumeSpecName: "dshm") pod "e95013af-8656-453e-baa5-4c7b9eeccb15" (UID: "e95013af-8656-453e-baa5-4c7b9eeccb15"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:24:16.385234 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.385145    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e95013af-8656-453e-baa5-4c7b9eeccb15-kube-api-access-xs8n2" (OuterVolumeSpecName: "kube-api-access-xs8n2") pod "e95013af-8656-453e-baa5-4c7b9eeccb15" (UID: "e95013af-8656-453e-baa5-4c7b9eeccb15"). InnerVolumeSpecName "kube-api-access-xs8n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:24:16.483904 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.483860    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:16.483904 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.483895    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:16.483904 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.483908    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs8n2\" (UniqueName: \"kubernetes.io/projected/e95013af-8656-453e-baa5-4c7b9eeccb15-kube-api-access-xs8n2\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:16.484140 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.483922    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95013af-8656-453e-baa5-4c7b9eeccb15-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:16.484140 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:16.483937    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95013af-8656-453e-baa5-4c7b9eeccb15-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:17.118118 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.118081    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"]
Jul 24 17:24:17.120557 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.120533    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-0a7def-bsnzk"]
Jul 24 17:24:17.192688 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.192658    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"
Jul 24 17:24:17.192853 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.192685    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8" event={"ID":"e95013af-8656-453e-baa5-4c7b9eeccb15","Type":"ContainerDied","Data":"50006b5770f8bda41093575c74eeed4046c90ae7a8590a1ef70c65c92af3bcb0"}
Jul 24 17:24:17.192853 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.192739    2569 scope.go:117] "RemoveContainer" containerID="0b45160e2f2e14176f15d26315aca3cb39d7289327e824713ef43806bcad38df"
Jul 24 17:24:17.216685 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.216644    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"]
Jul 24 17:24:17.218409 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.218388    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-0a7def/test-model-0a7def-kserve-7585d48ff9-49bl8"]
Jul 24 17:24:17.630295 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.630254    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17165019-aa97-48dd-a8e1-24e88a3142b6" path="/var/lib/kubelet/pods/17165019-aa97-48dd-a8e1-24e88a3142b6/volumes"
Jul 24 17:24:17.630699 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:17.630682    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e95013af-8656-453e-baa5-4c7b9eeccb15" path="/var/lib/kubelet/pods/e95013af-8656-453e-baa5-4c7b9eeccb15/volumes"
Jul 24 17:24:23.831134 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.831093    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"]
Jul 24 17:24:23.831767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.831483    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17165019-aa97-48dd-a8e1-24e88a3142b6" containerName="revoke-keys"
Jul 24 17:24:23.831767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.831503    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="17165019-aa97-48dd-a8e1-24e88a3142b6" containerName="revoke-keys"
Jul 24 17:24:23.831767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.831517    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e95013af-8656-453e-baa5-4c7b9eeccb15" containerName="main"
Jul 24 17:24:23.831767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.831523    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e95013af-8656-453e-baa5-4c7b9eeccb15" containerName="main"
Jul 24 17:24:23.831767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.831602    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="17165019-aa97-48dd-a8e1-24e88a3142b6" containerName="revoke-keys"
Jul 24 17:24:23.831767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.831611    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e95013af-8656-453e-baa5-4c7b9eeccb15" containerName="main"
Jul 24 17:24:23.834574 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.834553    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"
Jul 24 17:24:23.836696 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.836675    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:24:23.836812 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.836677    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:24:23.841068 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.841040    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"]
Jul 24 17:24:23.957892 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:23.957852    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaceed2-0940-440b-956c-6542893eb3b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf\" (UID: \"7eaceed2-0940-440b-956c-6542893eb3b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"
Jul 24 17:24:24.058785 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:24.058757    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaceed2-0940-440b-956c-6542893eb3b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf\" (UID: \"7eaceed2-0940-440b-956c-6542893eb3b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"
Jul 24 17:24:24.059403 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:24.059381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaceed2-0940-440b-956c-6542893eb3b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf\" (UID: \"7eaceed2-0940-440b-956c-6542893eb3b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"
Jul 24 17:24:24.146614 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:24.146535    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"
Jul 24 17:24:24.278871 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:24.278848    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"]
Jul 24 17:24:24.281562 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:24:24.281534    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eaceed2_0940_440b_956c_6542893eb3b7.slice/crio-f5d76e9e7cce1db17f2117e30c0d0bb2c652b3f4b7be0f02ad565f523dc5da12 WatchSource:0}: Error finding container f5d76e9e7cce1db17f2117e30c0d0bb2c652b3f4b7be0f02ad565f523dc5da12: Status 404 returned error can't find the container with id f5d76e9e7cce1db17f2117e30c0d0bb2c652b3f4b7be0f02ad565f523dc5da12
Jul 24 17:24:25.234814 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:25.234777    2569 generic.go:356] "Generic (PLEG): container finished" podID="7eaceed2-0940-440b-956c-6542893eb3b7" containerID="e8cda349a858e846d63342b34f15ec7ea50cab278768c732062c92413c94443b" exitCode=0
Jul 24 17:24:25.235214 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:25.234820    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf" event={"ID":"7eaceed2-0940-440b-956c-6542893eb3b7","Type":"ContainerDied","Data":"e8cda349a858e846d63342b34f15ec7ea50cab278768c732062c92413c94443b"}
Jul 24 17:24:25.235214 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:25.234871    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf" event={"ID":"7eaceed2-0940-440b-956c-6542893eb3b7","Type":"ContainerStarted","Data":"f5d76e9e7cce1db17f2117e30c0d0bb2c652b3f4b7be0f02ad565f523dc5da12"}
Jul 24 17:24:26.369662 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:26.369637    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"
Jul 24 17:24:26.485173 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:26.485141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaceed2-0940-440b-956c-6542893eb3b7-maas-api-service-ca\") pod \"7eaceed2-0940-440b-956c-6542893eb3b7\" (UID: \"7eaceed2-0940-440b-956c-6542893eb3b7\") "
Jul 24 17:24:26.485503 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:26.485480    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7eaceed2-0940-440b-956c-6542893eb3b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7eaceed2-0940-440b-956c-6542893eb3b7" (UID: "7eaceed2-0940-440b-956c-6542893eb3b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:24:26.586081 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:26.586048    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7eaceed2-0940-440b-956c-6542893eb3b7-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:24:27.245056 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:27.245025    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"
Jul 24 17:24:27.245230 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:27.245055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf" event={"ID":"7eaceed2-0940-440b-956c-6542893eb3b7","Type":"ContainerDied","Data":"f5d76e9e7cce1db17f2117e30c0d0bb2c652b3f4b7be0f02ad565f523dc5da12"}
Jul 24 17:24:27.245230 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:27.245102    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5d76e9e7cce1db17f2117e30c0d0bb2c652b3f4b7be0f02ad565f523dc5da12"
Jul 24 17:24:38.905294 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:38.905255    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"]
Jul 24 17:24:38.908408 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:38.908384    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b3a7b4-4vrkf"]
Jul 24 17:24:39.630276 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:24:39.630243    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eaceed2-0940-440b-956c-6542893eb3b7" path="/var/lib/kubelet/pods/7eaceed2-0940-440b-956c-6542893eb3b7/volumes"
Jul 24 17:25:09.963853 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.963770    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"]
Jul 24 17:25:09.964307 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.964128    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eaceed2-0940-440b-956c-6542893eb3b7" containerName="revoke-keys"
Jul 24 17:25:09.964307 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.964139    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eaceed2-0940-440b-956c-6542893eb3b7" containerName="revoke-keys"
Jul 24 17:25:09.964307 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.964203    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eaceed2-0940-440b-956c-6542893eb3b7" containerName="revoke-keys"
Jul 24 17:25:09.967460 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.967442    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"
Jul 24 17:25:09.969985 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.969961    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:25:09.970124 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.970042    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:25:09.975060 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:09.975038    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"]
Jul 24 17:25:10.073638 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:10.073598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37bbc30-f2c3-4672-976c-192028c7f920-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c87733-pmm8q\" (UID: \"c37bbc30-f2c3-4672-976c-192028c7f920\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"
Jul 24 17:25:10.174647 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:10.174610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37bbc30-f2c3-4672-976c-192028c7f920-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c87733-pmm8q\" (UID: \"c37bbc30-f2c3-4672-976c-192028c7f920\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"
Jul 24 17:25:10.175250 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:10.175228    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37bbc30-f2c3-4672-976c-192028c7f920-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c87733-pmm8q\" (UID: \"c37bbc30-f2c3-4672-976c-192028c7f920\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"
Jul 24 17:25:10.278583 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:10.278495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"
Jul 24 17:25:10.411607 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:10.411577    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"]
Jul 24 17:25:10.415431 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:25:10.415389    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37bbc30_f2c3_4672_976c_192028c7f920.slice/crio-933743b070ba0c399aedadc2355333de7004abc06e40355802dd2e9001171ed2 WatchSource:0}: Error finding container 933743b070ba0c399aedadc2355333de7004abc06e40355802dd2e9001171ed2: Status 404 returned error can't find the container with id 933743b070ba0c399aedadc2355333de7004abc06e40355802dd2e9001171ed2
Jul 24 17:25:10.455715 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:10.455690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q" event={"ID":"c37bbc30-f2c3-4672-976c-192028c7f920","Type":"ContainerStarted","Data":"933743b070ba0c399aedadc2355333de7004abc06e40355802dd2e9001171ed2"}
Jul 24 17:25:11.461974 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:11.461941    2569 generic.go:356] "Generic (PLEG): container finished" podID="c37bbc30-f2c3-4672-976c-192028c7f920" containerID="d708dd28516b3e57bc123a628a77061483bbeb28bb47132e83320f5ce47d5be3" exitCode=0
Jul 24 17:25:11.462414 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:11.461990    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q" event={"ID":"c37bbc30-f2c3-4672-976c-192028c7f920","Type":"ContainerDied","Data":"d708dd28516b3e57bc123a628a77061483bbeb28bb47132e83320f5ce47d5be3"}
Jul 24 17:25:12.595981 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:12.595952    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"
Jul 24 17:25:12.696800 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:12.696763    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37bbc30-f2c3-4672-976c-192028c7f920-maas-api-service-ca\") pod \"c37bbc30-f2c3-4672-976c-192028c7f920\" (UID: \"c37bbc30-f2c3-4672-976c-192028c7f920\") "
Jul 24 17:25:12.697160 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:12.697132    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37bbc30-f2c3-4672-976c-192028c7f920-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c37bbc30-f2c3-4672-976c-192028c7f920" (UID: "c37bbc30-f2c3-4672-976c-192028c7f920"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:25:12.797831 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:12.797746    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37bbc30-f2c3-4672-976c-192028c7f920-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:25:13.472065 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:13.472040    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"
Jul 24 17:25:13.472240 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:13.472039    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q" event={"ID":"c37bbc30-f2c3-4672-976c-192028c7f920","Type":"ContainerDied","Data":"933743b070ba0c399aedadc2355333de7004abc06e40355802dd2e9001171ed2"}
Jul 24 17:25:13.472240 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:13.472165    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="933743b070ba0c399aedadc2355333de7004abc06e40355802dd2e9001171ed2"
Jul 24 17:25:13.874761 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:13.874731    2569 scope.go:117] "RemoveContainer" containerID="7fe1d2493598f949729fec817da370756b70a48f4ef2e4621e6eac4f493ba785"
Jul 24 17:25:30.048500 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:30.048463    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"]
Jul 24 17:25:30.050945 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:30.050919    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c87733-pmm8q"]
Jul 24 17:25:31.630290 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:31.630253    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c37bbc30-f2c3-4672-976c-192028c7f920" path="/var/lib/kubelet/pods/c37bbc30-f2c3-4672-976c-192028c7f920/volumes"
Jul 24 17:25:36.768084 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.768040    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"]
Jul 24 17:25:36.768513 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.768418    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37bbc30-f2c3-4672-976c-192028c7f920" containerName="revoke-keys"
Jul 24 17:25:36.768513 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.768435    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37bbc30-f2c3-4672-976c-192028c7f920" containerName="revoke-keys"
Jul 24 17:25:36.768595 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.768517    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37bbc30-f2c3-4672-976c-192028c7f920" containerName="revoke-keys"
Jul 24 17:25:36.771860 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.771844    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"
Jul 24 17:25:36.774153 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.774119    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:25:36.774153 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.774138    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:25:36.778040 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.778016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"]
Jul 24 17:25:36.822298 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.822256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c241583-6ea1-4c43-9002-7b304d86830d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z\" (UID: \"5c241583-6ea1-4c43-9002-7b304d86830d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"
Jul 24 17:25:36.923426 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.923390    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c241583-6ea1-4c43-9002-7b304d86830d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z\" (UID: \"5c241583-6ea1-4c43-9002-7b304d86830d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"
Jul 24 17:25:36.924026 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:36.924005    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c241583-6ea1-4c43-9002-7b304d86830d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z\" (UID: \"5c241583-6ea1-4c43-9002-7b304d86830d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"
Jul 24 17:25:37.084065 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:37.084035    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"
Jul 24 17:25:37.221609 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:37.221560    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"]
Jul 24 17:25:37.224602 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:25:37.224567    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c241583_6ea1_4c43_9002_7b304d86830d.slice/crio-a03ebbfa1711822a15e5514ddf19ba059d4d306d0cadb9fec7a60a130d1bbad1 WatchSource:0}: Error finding container a03ebbfa1711822a15e5514ddf19ba059d4d306d0cadb9fec7a60a130d1bbad1: Status 404 returned error can't find the container with id a03ebbfa1711822a15e5514ddf19ba059d4d306d0cadb9fec7a60a130d1bbad1
Jul 24 17:25:37.580759 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:37.580724    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z" event={"ID":"5c241583-6ea1-4c43-9002-7b304d86830d","Type":"ContainerStarted","Data":"2b1ecf1114494efcfebb06e059f437c66ce3e04c3e5fb4bfcb3c974a6fa6aecb"}
Jul 24 17:25:37.580958 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:37.580773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z" event={"ID":"5c241583-6ea1-4c43-9002-7b304d86830d","Type":"ContainerStarted","Data":"a03ebbfa1711822a15e5514ddf19ba059d4d306d0cadb9fec7a60a130d1bbad1"}
Jul 24 17:25:37.597271 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:37.597218    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z" podStartSLOduration=1.597201567 podStartE2EDuration="1.597201567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:25:37.59570628 +0000 UTC m=+2544.355515278" watchObservedRunningTime="2026-07-24 17:25:37.597201567 +0000 UTC m=+2544.357010594"
Jul 24 17:25:38.586571 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:38.586535    2569 generic.go:356] "Generic (PLEG): container finished" podID="5c241583-6ea1-4c43-9002-7b304d86830d" containerID="2b1ecf1114494efcfebb06e059f437c66ce3e04c3e5fb4bfcb3c974a6fa6aecb" exitCode=0
Jul 24 17:25:38.587015 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:38.586616    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z" event={"ID":"5c241583-6ea1-4c43-9002-7b304d86830d","Type":"ContainerDied","Data":"2b1ecf1114494efcfebb06e059f437c66ce3e04c3e5fb4bfcb3c974a6fa6aecb"}
Jul 24 17:25:39.723090 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:39.723062    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"
Jul 24 17:25:39.850875 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:39.850783    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c241583-6ea1-4c43-9002-7b304d86830d-maas-api-service-ca\") pod \"5c241583-6ea1-4c43-9002-7b304d86830d\" (UID: \"5c241583-6ea1-4c43-9002-7b304d86830d\") "
Jul 24 17:25:39.851158 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:39.851134    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c241583-6ea1-4c43-9002-7b304d86830d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c241583-6ea1-4c43-9002-7b304d86830d" (UID: "5c241583-6ea1-4c43-9002-7b304d86830d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:25:39.951723 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:39.951690    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c241583-6ea1-4c43-9002-7b304d86830d-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:25:40.597490 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:40.597457    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"
Jul 24 17:25:40.597700 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:40.597489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z" event={"ID":"5c241583-6ea1-4c43-9002-7b304d86830d","Type":"ContainerDied","Data":"a03ebbfa1711822a15e5514ddf19ba059d4d306d0cadb9fec7a60a130d1bbad1"}
Jul 24 17:25:40.597700 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:40.597572    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a03ebbfa1711822a15e5514ddf19ba059d4d306d0cadb9fec7a60a130d1bbad1"
Jul 24 17:25:56.845933 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:56.845895    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"]
Jul 24 17:25:56.848472 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:56.848447    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ba18-8mp6z"]
Jul 24 17:25:57.631925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:25:57.631890    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c241583-6ea1-4c43-9002-7b304d86830d" path="/var/lib/kubelet/pods/5c241583-6ea1-4c43-9002-7b304d86830d/volumes"
Jul 24 17:26:40.286045 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.285948    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"]
Jul 24 17:26:40.288537 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.286362    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c241583-6ea1-4c43-9002-7b304d86830d" containerName="revoke-keys"
Jul 24 17:26:40.288537 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.286377    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c241583-6ea1-4c43-9002-7b304d86830d" containerName="revoke-keys"
Jul 24 17:26:40.288537 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.286441    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c241583-6ea1-4c43-9002-7b304d86830d" containerName="revoke-keys"
Jul 24 17:26:40.289447 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.289433    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.291883 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.291866    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"kube-root-ca.crt\""
Jul 24 17:26:40.292767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.292746    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"default-dockercfg-sskzd\""
Jul 24 17:26:40.292922 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.292819    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"openshift-service-ca.crt\""
Jul 24 17:26:40.292922 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.292822    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"auth-test-model-de68c7-kserve-self-signed-certs\""
Jul 24 17:26:40.294169 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.294149    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-dshm\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.294261 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.294188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-model-cache\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.294261 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.294235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ab95e-0a13-4bb4-b609-330414ffa32b-tls-certs\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.294433 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.294263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-tmp-dir\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.294433 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.294294    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-home\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.294433 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.294311    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zh2m\" (UniqueName: \"kubernetes.io/projected/628ab95e-0a13-4bb4-b609-330414ffa32b-kube-api-access-6zh2m\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.298931 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.298912    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"]
Jul 24 17:26:40.395037 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.394995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-home\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395256 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zh2m\" (UniqueName: \"kubernetes.io/projected/628ab95e-0a13-4bb4-b609-330414ffa32b-kube-api-access-6zh2m\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395256 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395093    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-dshm\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395256 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395142    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-model-cache\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395256 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395180    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ab95e-0a13-4bb4-b609-330414ffa32b-tls-certs\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395256 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-tmp-dir\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395564 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395451    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-home\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395564 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395516    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-model-cache\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.395667 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.395616    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-tmp-dir\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.397387 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.397359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-dshm\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.397648 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.397630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ab95e-0a13-4bb4-b609-330414ffa32b-tls-certs\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.402924 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.402899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zh2m\" (UniqueName: \"kubernetes.io/projected/628ab95e-0a13-4bb4-b609-330414ffa32b-kube-api-access-6zh2m\") pod \"auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") " pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.599274 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.599237    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:40.742626 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.742599    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"]
Jul 24 17:26:40.745018 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:26:40.744988    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod628ab95e_0a13_4bb4_b609_330414ffa32b.slice/crio-66ba5b7bd10b69dc7e636267c615fac7e7691d80941a404700630464f9bcff83 WatchSource:0}: Error finding container 66ba5b7bd10b69dc7e636267c615fac7e7691d80941a404700630464f9bcff83: Status 404 returned error can't find the container with id 66ba5b7bd10b69dc7e636267c615fac7e7691d80941a404700630464f9bcff83
Jul 24 17:26:40.890857 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.890761    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw" event={"ID":"628ab95e-0a13-4bb4-b609-330414ffa32b","Type":"ContainerStarted","Data":"614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da"}
Jul 24 17:26:40.890857 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.890804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw" event={"ID":"628ab95e-0a13-4bb4-b609-330414ffa32b","Type":"ContainerStarted","Data":"66ba5b7bd10b69dc7e636267c615fac7e7691d80941a404700630464f9bcff83"}
Jul 24 17:26:40.908187 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:40.908137    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw" podStartSLOduration=0.908120781 podStartE2EDuration="908.120781ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:26:40.907552921 +0000 UTC m=+2607.667361921" watchObservedRunningTime="2026-07-24 17:26:40.908120781 +0000 UTC m=+2607.667929825"
Jul 24 17:26:50.599604 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:50.599559    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:50.600153 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:50.599629    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:50.612060 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:50.612032    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:50.956764 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:50.956684    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:26:55.572769 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.572736    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"]
Jul 24 17:26:55.576103 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.576086    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.578481 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.578458    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"kube-root-ca.crt\""
Jul 24 17:26:55.579266 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.579250    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"openshift-service-ca.crt\""
Jul 24 17:26:55.579415 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.579398    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"auth-test-model-d73e95-kserve-self-signed-certs\""
Jul 24 17:26:55.579497 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.579434    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"default-dockercfg-2f7c7\""
Jul 24 17:26:55.585158 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.585135    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"]
Jul 24 17:26:55.621900 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.621864    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tmp-dir\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.621900 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.621906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-model-cache\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.622114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.622005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-dshm\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.622114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.622043    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-home\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.622114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.622065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tls-certs\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.622114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.622081    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwhtg\" (UniqueName: \"kubernetes.io/projected/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-kube-api-access-nwhtg\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.722701 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.722662    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-dshm\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.722701 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.722701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-home\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.722975 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.722718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tls-certs\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.722975 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.722734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwhtg\" (UniqueName: \"kubernetes.io/projected/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-kube-api-access-nwhtg\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.722975 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.722806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tmp-dir\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.722975 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.722842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-model-cache\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.723206 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.723174    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-home\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.723306 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.723207    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-model-cache\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.723430 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.723409    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tmp-dir\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.725098 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.725075    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-dshm\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.725389 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.725370    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tls-certs\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.731063 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.731039    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwhtg\" (UniqueName: \"kubernetes.io/projected/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-kube-api-access-nwhtg\") pod \"auth-test-model-d73e95-kserve-9bfffc57f-6dnt6\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") " pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:55.886434 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:55.886313    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:26:56.022296 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:56.022270    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"]
Jul 24 17:26:56.024462 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:26:56.024435    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a689e70_5a2f_48cb_a967_78bbd5ea7e77.slice/crio-ce776a7246ddf829946377cd58a7f223dc4f27c52a53ecdf03b4b0483b412719 WatchSource:0}: Error finding container ce776a7246ddf829946377cd58a7f223dc4f27c52a53ecdf03b4b0483b412719: Status 404 returned error can't find the container with id ce776a7246ddf829946377cd58a7f223dc4f27c52a53ecdf03b4b0483b412719
Jul 24 17:26:56.975059 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:56.975019    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6" event={"ID":"1a689e70-5a2f-48cb-a967-78bbd5ea7e77","Type":"ContainerStarted","Data":"6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d"}
Jul 24 17:26:56.975451 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:56.975069    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6" event={"ID":"1a689e70-5a2f-48cb-a967-78bbd5ea7e77","Type":"ContainerStarted","Data":"ce776a7246ddf829946377cd58a7f223dc4f27c52a53ecdf03b4b0483b412719"}
Jul 24 17:26:56.990644 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:26:56.990596    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6" podStartSLOduration=1.990580781 podStartE2EDuration="1.990580781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:26:56.990155815 +0000 UTC m=+2623.749964830" watchObservedRunningTime="2026-07-24 17:26:56.990580781 +0000 UTC m=+2623.750389781"
Jul 24 17:27:05.887348 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:27:05.887287    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:27:05.887802 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:27:05.887368    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:27:05.899829 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:27:05.899804    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:27:06.031283 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:27:06.031252    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:27:13.904575 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:27:13.904545    2569 scope.go:117] "RemoveContainer" containerID="5728d6311861c296b583442777b7d70afa342cb3d949bc0e51401d68df2c882e"
Jul 24 17:29:03.943282 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:03.943249    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"]
Jul 24 17:29:04.863751 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:04.863709    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"]
Jul 24 17:29:18.962151 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:18.962088    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw" podUID="628ab95e-0a13-4bb4-b609-330414ffa32b" containerName="main" containerID="cri-o://614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da" gracePeriod=45
Jul 24 17:29:19.211052 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.211026    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:29:19.358868 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.358836    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-dshm\") pod \"628ab95e-0a13-4bb4-b609-330414ffa32b\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") "
Jul 24 17:29:19.359127 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.358888    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-model-cache\") pod \"628ab95e-0a13-4bb4-b609-330414ffa32b\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") "
Jul 24 17:29:19.359127 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.358911    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-home\") pod \"628ab95e-0a13-4bb4-b609-330414ffa32b\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") "
Jul 24 17:29:19.359127 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.358948    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zh2m\" (UniqueName: \"kubernetes.io/projected/628ab95e-0a13-4bb4-b609-330414ffa32b-kube-api-access-6zh2m\") pod \"628ab95e-0a13-4bb4-b609-330414ffa32b\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") "
Jul 24 17:29:19.359127 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.359009    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-tmp-dir\") pod \"628ab95e-0a13-4bb4-b609-330414ffa32b\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") "
Jul 24 17:29:19.359127 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.359046    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ab95e-0a13-4bb4-b609-330414ffa32b-tls-certs\") pod \"628ab95e-0a13-4bb4-b609-330414ffa32b\" (UID: \"628ab95e-0a13-4bb4-b609-330414ffa32b\") "
Jul 24 17:29:19.359431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.359170    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-model-cache" (OuterVolumeSpecName: "model-cache") pod "628ab95e-0a13-4bb4-b609-330414ffa32b" (UID: "628ab95e-0a13-4bb4-b609-330414ffa32b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:19.359431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.359237    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-home" (OuterVolumeSpecName: "home") pod "628ab95e-0a13-4bb4-b609-330414ffa32b" (UID: "628ab95e-0a13-4bb4-b609-330414ffa32b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:19.359431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.359301    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "628ab95e-0a13-4bb4-b609-330414ffa32b" (UID: "628ab95e-0a13-4bb4-b609-330414ffa32b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:19.359431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.359383    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:19.359431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.359397    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:19.361209 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.361182    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-dshm" (OuterVolumeSpecName: "dshm") pod "628ab95e-0a13-4bb4-b609-330414ffa32b" (UID: "628ab95e-0a13-4bb4-b609-330414ffa32b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:19.361209 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.361202    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/628ab95e-0a13-4bb4-b609-330414ffa32b-kube-api-access-6zh2m" (OuterVolumeSpecName: "kube-api-access-6zh2m") pod "628ab95e-0a13-4bb4-b609-330414ffa32b" (UID: "628ab95e-0a13-4bb4-b609-330414ffa32b"). InnerVolumeSpecName "kube-api-access-6zh2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:29:19.361384 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.361273    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/628ab95e-0a13-4bb4-b609-330414ffa32b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "628ab95e-0a13-4bb4-b609-330414ffa32b" (UID: "628ab95e-0a13-4bb4-b609-330414ffa32b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:29:19.460179 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.460147    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:19.460179 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.460173    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/628ab95e-0a13-4bb4-b609-330414ffa32b-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:19.460179 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.460183    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/628ab95e-0a13-4bb4-b609-330414ffa32b-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:19.460429 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.460192    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zh2m\" (UniqueName: \"kubernetes.io/projected/628ab95e-0a13-4bb4-b609-330414ffa32b-kube-api-access-6zh2m\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:19.668455 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.668419    2569 generic.go:356] "Generic (PLEG): container finished" podID="628ab95e-0a13-4bb4-b609-330414ffa32b" containerID="614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da" exitCode=0
Jul 24 17:29:19.668625 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.668460    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"
Jul 24 17:29:19.668625 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.668490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw" event={"ID":"628ab95e-0a13-4bb4-b609-330414ffa32b","Type":"ContainerDied","Data":"614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da"}
Jul 24 17:29:19.668625 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.668526    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw" event={"ID":"628ab95e-0a13-4bb4-b609-330414ffa32b","Type":"ContainerDied","Data":"66ba5b7bd10b69dc7e636267c615fac7e7691d80941a404700630464f9bcff83"}
Jul 24 17:29:19.668625 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.668546    2569 scope.go:117] "RemoveContainer" containerID="614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da"
Jul 24 17:29:19.677034 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.677013    2569 scope.go:117] "RemoveContainer" containerID="614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da"
Jul 24 17:29:19.677290 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:29:19.677272    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da\": container with ID starting with 614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da not found: ID does not exist" containerID="614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da"
Jul 24 17:29:19.677386 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.677302    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da"} err="failed to get container status \"614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da\": rpc error: code = NotFound desc = could not find container \"614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da\": container with ID starting with 614abbedb02995d6d9249ca7ec4002957738b894084e0e0393fe3e421eeb87da not found: ID does not exist"
Jul 24 17:29:19.685202 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.685180    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"]
Jul 24 17:29:19.689259 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.689238    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/auth-test-model-de68c7-kserve-7f8969f9f5-ms7mw"]
Jul 24 17:29:19.882756 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:19.882719    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6" podUID="1a689e70-5a2f-48cb-a967-78bbd5ea7e77" containerName="main" containerID="cri-o://6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d" gracePeriod=45
Jul 24 17:29:20.144471 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.144447    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:29:20.268239 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268138    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-home\") pod \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") "
Jul 24 17:29:20.268239 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268216    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-model-cache\") pod \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") "
Jul 24 17:29:20.268506 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268257    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tls-certs\") pod \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") "
Jul 24 17:29:20.268506 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268273    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-dshm\") pod \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") "
Jul 24 17:29:20.268506 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268298    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwhtg\" (UniqueName: \"kubernetes.io/projected/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-kube-api-access-nwhtg\") pod \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") "
Jul 24 17:29:20.268506 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268371    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tmp-dir\") pod \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\" (UID: \"1a689e70-5a2f-48cb-a967-78bbd5ea7e77\") "
Jul 24 17:29:20.268506 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-home" (OuterVolumeSpecName: "home") pod "1a689e70-5a2f-48cb-a967-78bbd5ea7e77" (UID: "1a689e70-5a2f-48cb-a967-78bbd5ea7e77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:20.268766 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268586    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a689e70-5a2f-48cb-a967-78bbd5ea7e77" (UID: "1a689e70-5a2f-48cb-a967-78bbd5ea7e77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:20.268766 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268655    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a689e70-5a2f-48cb-a967-78bbd5ea7e77" (UID: "1a689e70-5a2f-48cb-a967-78bbd5ea7e77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:20.268766 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268717    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:20.268766 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.268736    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:20.270537 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.270508    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-dshm" (OuterVolumeSpecName: "dshm") pod "1a689e70-5a2f-48cb-a967-78bbd5ea7e77" (UID: "1a689e70-5a2f-48cb-a967-78bbd5ea7e77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:29:20.270883 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.270860    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a689e70-5a2f-48cb-a967-78bbd5ea7e77" (UID: "1a689e70-5a2f-48cb-a967-78bbd5ea7e77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:29:20.270956 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.270890    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-kube-api-access-nwhtg" (OuterVolumeSpecName: "kube-api-access-nwhtg") pod "1a689e70-5a2f-48cb-a967-78bbd5ea7e77" (UID: "1a689e70-5a2f-48cb-a967-78bbd5ea7e77"). InnerVolumeSpecName "kube-api-access-nwhtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:29:20.369450 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.369412    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwhtg\" (UniqueName: \"kubernetes.io/projected/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-kube-api-access-nwhtg\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:20.369450 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.369438    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:20.369450 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.369450    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:20.369450 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.369458    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a689e70-5a2f-48cb-a967-78bbd5ea7e77-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:29:20.675499 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.675466    2569 generic.go:356] "Generic (PLEG): container finished" podID="1a689e70-5a2f-48cb-a967-78bbd5ea7e77" containerID="6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d" exitCode=0
Jul 24 17:29:20.675685 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.675533    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"
Jul 24 17:29:20.675685 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.675561    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6" event={"ID":"1a689e70-5a2f-48cb-a967-78bbd5ea7e77","Type":"ContainerDied","Data":"6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d"}
Jul 24 17:29:20.675685 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.675610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6" event={"ID":"1a689e70-5a2f-48cb-a967-78bbd5ea7e77","Type":"ContainerDied","Data":"ce776a7246ddf829946377cd58a7f223dc4f27c52a53ecdf03b4b0483b412719"}
Jul 24 17:29:20.675685 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.675628    2569 scope.go:117] "RemoveContainer" containerID="6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d"
Jul 24 17:29:20.684666 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.684650    2569 scope.go:117] "RemoveContainer" containerID="6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d"
Jul 24 17:29:20.684944 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:29:20.684927    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d\": container with ID starting with 6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d not found: ID does not exist" containerID="6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d"
Jul 24 17:29:20.685019 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.684949    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d"} err="failed to get container status \"6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d\": rpc error: code = NotFound desc = could not find container \"6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d\": container with ID starting with 6aaecee8096de75e4c7dd788b7ca831f990c8ef5c6a7adecbc26f82ea8127c9d not found: ID does not exist"
Jul 24 17:29:20.696129 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.696107    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"]
Jul 24 17:29:20.699968 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:20.699949    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/auth-test-model-d73e95-kserve-9bfffc57f-6dnt6"]
Jul 24 17:29:21.630210 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:21.630168    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a689e70-5a2f-48cb-a967-78bbd5ea7e77" path="/var/lib/kubelet/pods/1a689e70-5a2f-48cb-a967-78bbd5ea7e77/volumes"
Jul 24 17:29:21.630794 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:21.630773    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="628ab95e-0a13-4bb4-b609-330414ffa32b" path="/var/lib/kubelet/pods/628ab95e-0a13-4bb4-b609-330414ffa32b/volumes"
Jul 24 17:29:30.190036 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.190003    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"]
Jul 24 17:29:30.190512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.190361    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="628ab95e-0a13-4bb4-b609-330414ffa32b" containerName="main"
Jul 24 17:29:30.190512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.190373    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="628ab95e-0a13-4bb4-b609-330414ffa32b" containerName="main"
Jul 24 17:29:30.190512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.190387    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a689e70-5a2f-48cb-a967-78bbd5ea7e77" containerName="main"
Jul 24 17:29:30.190512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.190394    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a689e70-5a2f-48cb-a967-78bbd5ea7e77" containerName="main"
Jul 24 17:29:30.190512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.190444    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a689e70-5a2f-48cb-a967-78bbd5ea7e77" containerName="main"
Jul 24 17:29:30.190512 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.190455    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="628ab95e-0a13-4bb4-b609-330414ffa32b" containerName="main"
Jul 24 17:29:30.192944 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.192927    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.195437 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.195412    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"kube-root-ca.crt\""
Jul 24 17:29:30.196353 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.196317    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"default-dockercfg-sskzd\""
Jul 24 17:29:30.196476 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.196357    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"openshift-service-ca.crt\""
Jul 24 17:29:30.196476 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.196409    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-de68c7\"/\"sub-test-model-de68c7-kserve-self-signed-certs\""
Jul 24 17:29:30.203049 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.203027    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"]
Jul 24 17:29:30.262901 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.262867    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-home\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.263071 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.262915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-model-cache\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.263071 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.262935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tls-certs\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.263071 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.262951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tmp-dir\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.263071 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.262982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-dshm\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.263071 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.263007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xr79\" (UniqueName: \"kubernetes.io/projected/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-kube-api-access-2xr79\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.363800 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.363766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-model-cache\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.363800 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.363803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tls-certs\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.364054 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.363820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tmp-dir\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.364054 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.363858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-dshm\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.364054 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.363883    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xr79\" (UniqueName: \"kubernetes.io/projected/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-kube-api-access-2xr79\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.364054 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.363928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-home\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.364255 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.364216    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-model-cache\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.364301 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.364279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tmp-dir\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.364386 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.364348    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-home\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.366126 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.366103    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-dshm\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.366402 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.366386    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tls-certs\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.371202 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.371183    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xr79\" (UniqueName: \"kubernetes.io/projected/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-kube-api-access-2xr79\") pod \"sub-test-model-de68c7-kserve-7859cc9985-zxdkl\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") " pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.501867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.501765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:30.632045 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.632019    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"]
Jul 24 17:29:30.634360 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:29:30.634298    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod415e9c8e_92c3_4ce5_8ad1_c723dcc8c84b.slice/crio-334d08ed1d0e9fdfe9695e8be8d05667744d683d982f1d17e00a4c2acc6b8e24 WatchSource:0}: Error finding container 334d08ed1d0e9fdfe9695e8be8d05667744d683d982f1d17e00a4c2acc6b8e24: Status 404 returned error can't find the container with id 334d08ed1d0e9fdfe9695e8be8d05667744d683d982f1d17e00a4c2acc6b8e24
Jul 24 17:29:30.725712 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.725682    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl" event={"ID":"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b","Type":"ContainerStarted","Data":"08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed"}
Jul 24 17:29:30.725894 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.725722    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl" event={"ID":"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b","Type":"ContainerStarted","Data":"334d08ed1d0e9fdfe9695e8be8d05667744d683d982f1d17e00a4c2acc6b8e24"}
Jul 24 17:29:30.742411 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:30.742353    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl" podStartSLOduration=0.742319455 podStartE2EDuration="742.319455ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:29:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:29:30.741641016 +0000 UTC m=+2777.501450016" watchObservedRunningTime="2026-07-24 17:29:30.742319455 +0000 UTC m=+2777.502128455"
Jul 24 17:29:40.502140 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:40.502062    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:40.502140 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:40.502106    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:40.514634 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:40.514608    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:29:40.786018 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:29:40.785942    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:30:00.140130 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.140089    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l"]
Jul 24 17:30:00.143662 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.143634    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"]
Jul 24 17:30:00.143803 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.143732    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l"
Jul 24 17:30:00.146078 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.146059    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:30:00.146896 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.146881    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"]
Jul 24 17:30:00.146991 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.146977    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"
Jul 24 17:30:00.149966 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.149939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"
Jul 24 17:30:00.152973 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.152952    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l"]
Jul 24 17:30:00.155676 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.155653    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"]
Jul 24 17:30:00.157283 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.157264    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l"
Jul 24 17:30:00.157803 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.157782    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"]
Jul 24 17:30:00.167498 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.167478    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"
Jul 24 17:30:00.204892 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.204855    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"
Jul 24 17:30:00.317088 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.317058    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l"]
Jul 24 17:30:00.319457 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:30:00.319070    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8517db9c_904c_4834_95fd_1b9c7380612c.slice/crio-4366e565768c3f5e6a6651b40ab7367a923e19c91591a69b160802270b332dca WatchSource:0}: Error finding container 4366e565768c3f5e6a6651b40ab7367a923e19c91591a69b160802270b332dca: Status 404 returned error can't find the container with id 4366e565768c3f5e6a6651b40ab7367a923e19c91591a69b160802270b332dca
Jul 24 17:30:00.369319 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.369296    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"]
Jul 24 17:30:00.371387 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:30:00.371364    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91da7613_6582_4fae_8c4b_116d58ba70f4.slice/crio-59b1cd1c8278f148365a231cb4c8d44815520f43b4bb25b086567d770d606f61 WatchSource:0}: Error finding container 59b1cd1c8278f148365a231cb4c8d44815520f43b4bb25b086567d770d606f61: Status 404 returned error can't find the container with id 59b1cd1c8278f148365a231cb4c8d44815520f43b4bb25b086567d770d606f61
Jul 24 17:30:00.553437 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.553411    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"]
Jul 24 17:30:00.555186 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:30:00.555163    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748df225_a88f_4049_ae58_3a0f4fb92e7d.slice/crio-7f19a47ab1c632f658c8526f8c28c1b97f06eefaa52e7353c7bc9e0e1aa10062 WatchSource:0}: Error finding container 7f19a47ab1c632f658c8526f8c28c1b97f06eefaa52e7353c7bc9e0e1aa10062: Status 404 returned error can't find the container with id 7f19a47ab1c632f658c8526f8c28c1b97f06eefaa52e7353c7bc9e0e1aa10062
Jul 24 17:30:00.866004 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.865966    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l" event={"ID":"8517db9c-904c-4834-95fd-1b9c7380612c","Type":"ContainerStarted","Data":"7a7d5655a218b85b316a7eb5b7cf16b81cab68d56bb55acda21832b576e79528"}
Jul 24 17:30:00.866219 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.866036    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l" event={"ID":"8517db9c-904c-4834-95fd-1b9c7380612c","Type":"ContainerStarted","Data":"4366e565768c3f5e6a6651b40ab7367a923e19c91591a69b160802270b332dca"}
Jul 24 17:30:00.867538 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.867514    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45" event={"ID":"91da7613-6582-4fae-8c4b-116d58ba70f4","Type":"ContainerStarted","Data":"70f7e289c1a01922648adba4fa7c58f5a7668dec365acefc83df6793fa3f9507"}
Jul 24 17:30:00.867677 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.867555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45" event={"ID":"91da7613-6582-4fae-8c4b-116d58ba70f4","Type":"ContainerStarted","Data":"59b1cd1c8278f148365a231cb4c8d44815520f43b4bb25b086567d770d606f61"}
Jul 24 17:30:00.869061 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.869036    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9" event={"ID":"748df225-a88f-4049-ae58-3a0f4fb92e7d","Type":"ContainerStarted","Data":"b74647bda8f7106921eaf9428a9d03a8347164508360ad49dfe10e506e50137f"}
Jul 24 17:30:00.869162 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.869075    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9" event={"ID":"748df225-a88f-4049-ae58-3a0f4fb92e7d","Type":"ContainerStarted","Data":"7f19a47ab1c632f658c8526f8c28c1b97f06eefaa52e7353c7bc9e0e1aa10062"}
Jul 24 17:30:00.880669 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.880622    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l" podStartSLOduration=0.880608354 podStartE2EDuration="880.608354ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:00.879300358 +0000 UTC m=+2807.639109359" watchObservedRunningTime="2026-07-24 17:30:00.880608354 +0000 UTC m=+2807.640417353"
Jul 24 17:30:00.893585 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.893542    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45" podStartSLOduration=0.893528586 podStartE2EDuration="893.528586ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:00.892461684 +0000 UTC m=+2807.652270682" watchObservedRunningTime="2026-07-24 17:30:00.893528586 +0000 UTC m=+2807.653337642"
Jul 24 17:30:00.908301 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:00.908252    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9" podStartSLOduration=0.908238505 podStartE2EDuration="908.238505ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:00.907074747 +0000 UTC m=+2807.666883746" watchObservedRunningTime="2026-07-24 17:30:00.908238505 +0000 UTC m=+2807.668047519"
Jul 24 17:30:01.875346 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:01.875291    2569 generic.go:356] "Generic (PLEG): container finished" podID="8517db9c-904c-4834-95fd-1b9c7380612c" containerID="7a7d5655a218b85b316a7eb5b7cf16b81cab68d56bb55acda21832b576e79528" exitCode=0
Jul 24 17:30:01.875819 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:01.875363    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l" event={"ID":"8517db9c-904c-4834-95fd-1b9c7380612c","Type":"ContainerDied","Data":"7a7d5655a218b85b316a7eb5b7cf16b81cab68d56bb55acda21832b576e79528"}
Jul 24 17:30:01.877035 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:01.877009    2569 generic.go:356] "Generic (PLEG): container finished" podID="91da7613-6582-4fae-8c4b-116d58ba70f4" containerID="70f7e289c1a01922648adba4fa7c58f5a7668dec365acefc83df6793fa3f9507" exitCode=0
Jul 24 17:30:01.877129 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:01.877094    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45" event={"ID":"91da7613-6582-4fae-8c4b-116d58ba70f4","Type":"ContainerDied","Data":"70f7e289c1a01922648adba4fa7c58f5a7668dec365acefc83df6793fa3f9507"}
Jul 24 17:30:01.878398 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:01.878373    2569 generic.go:356] "Generic (PLEG): container finished" podID="748df225-a88f-4049-ae58-3a0f4fb92e7d" containerID="b74647bda8f7106921eaf9428a9d03a8347164508360ad49dfe10e506e50137f" exitCode=0
Jul 24 17:30:01.878496 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:01.878421    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9" event={"ID":"748df225-a88f-4049-ae58-3a0f4fb92e7d","Type":"ContainerDied","Data":"b74647bda8f7106921eaf9428a9d03a8347164508360ad49dfe10e506e50137f"}
Jul 24 17:30:03.019447 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.019416    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"
Jul 24 17:30:03.061487 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.061457    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l"
Jul 24 17:30:03.064457 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.064438    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"
Jul 24 17:30:03.889076 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.889043    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"
Jul 24 17:30:03.889260 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.889077    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9" event={"ID":"748df225-a88f-4049-ae58-3a0f4fb92e7d","Type":"ContainerDied","Data":"7f19a47ab1c632f658c8526f8c28c1b97f06eefaa52e7353c7bc9e0e1aa10062"}
Jul 24 17:30:03.889260 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.889128    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f19a47ab1c632f658c8526f8c28c1b97f06eefaa52e7353c7bc9e0e1aa10062"
Jul 24 17:30:03.890581 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.890522    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l"
Jul 24 17:30:03.890581 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.890562    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-8fc7l" event={"ID":"8517db9c-904c-4834-95fd-1b9c7380612c","Type":"ContainerDied","Data":"4366e565768c3f5e6a6651b40ab7367a923e19c91591a69b160802270b332dca"}
Jul 24 17:30:03.890752 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.890601    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4366e565768c3f5e6a6651b40ab7367a923e19c91591a69b160802270b332dca"
Jul 24 17:30:03.892019 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.891999    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45" event={"ID":"91da7613-6582-4fae-8c4b-116d58ba70f4","Type":"ContainerDied","Data":"59b1cd1c8278f148365a231cb4c8d44815520f43b4bb25b086567d770d606f61"}
Jul 24 17:30:03.892019 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.892015    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"
Jul 24 17:30:03.892137 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:03.892039    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59b1cd1c8278f148365a231cb4c8d44815520f43b4bb25b086567d770d606f61"
Jul 24 17:30:13.941511 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:13.941483    2569 scope.go:117] "RemoveContainer" containerID="d07ca2747252a45a17ed5e56908d341d66c73595ad4643762383b70f3c3101a1"
Jul 24 17:30:14.372449 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372412    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"]
Jul 24 17:30:14.372768 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372756    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91da7613-6582-4fae-8c4b-116d58ba70f4" containerName="cleanup"
Jul 24 17:30:14.372817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372769    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="91da7613-6582-4fae-8c4b-116d58ba70f4" containerName="cleanup"
Jul 24 17:30:14.372817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372785    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748df225-a88f-4049-ae58-3a0f4fb92e7d" containerName="cleanup"
Jul 24 17:30:14.372817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372791    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="748df225-a88f-4049-ae58-3a0f4fb92e7d" containerName="cleanup"
Jul 24 17:30:14.372817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372800    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8517db9c-904c-4834-95fd-1b9c7380612c" containerName="cleanup"
Jul 24 17:30:14.372817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372805    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8517db9c-904c-4834-95fd-1b9c7380612c" containerName="cleanup"
Jul 24 17:30:14.372972 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372865    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="748df225-a88f-4049-ae58-3a0f4fb92e7d" containerName="cleanup"
Jul 24 17:30:14.372972 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372872    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8517db9c-904c-4834-95fd-1b9c7380612c" containerName="cleanup"
Jul 24 17:30:14.372972 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.372880    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="91da7613-6582-4fae-8c4b-116d58ba70f4" containerName="cleanup"
Jul 24 17:30:14.377226 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.377207    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.379945 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.379921    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"kube-root-ca.crt\""
Jul 24 17:30:14.379945 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.379944    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"sub-test-model-d73e95-kserve-self-signed-certs\""
Jul 24 17:30:14.380885 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.380860    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"default-dockercfg-2f7c7\""
Jul 24 17:30:14.381015 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.380897    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"openshift-service-ca.crt\""
Jul 24 17:30:14.386811 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.386786    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"]
Jul 24 17:30:14.570767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.570728    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-model-cache\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.570767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.570770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tmp-dir\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.570983 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.570795    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tls-certs\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.570983 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.570877    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-home\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.570983 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.570926    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csnw6\" (UniqueName: \"kubernetes.io/projected/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-kube-api-access-csnw6\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.570983 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.570957    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-dshm\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.672563 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.672466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-home\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.672563 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.672530    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csnw6\" (UniqueName: \"kubernetes.io/projected/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-kube-api-access-csnw6\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.672791 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.672576    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-dshm\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.672791 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.672596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-model-cache\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.672791 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.672620    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tmp-dir\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.672791 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.672656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tls-certs\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.673009 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.672945    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-home\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.673068 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.673026    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-model-cache\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.673119 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.673100    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tmp-dir\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.674828 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.674803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-dshm\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.675116 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.675099    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tls-certs\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.680731 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.680705    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csnw6\" (UniqueName: \"kubernetes.io/projected/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-kube-api-access-csnw6\") pod \"sub-test-model-d73e95-kserve-75cbf7df69-4sf92\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") " pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.688121 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.688098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:14.817879 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.817855    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"]
Jul 24 17:30:14.819585 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:30:14.819554    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd191876f_0f3c_4526_bb8a_b8ced28e9dc1.slice/crio-f401434bbccf7ac9ae0af7d6b985905d384a899428d8bc721f011e91ff3b6b98 WatchSource:0}: Error finding container f401434bbccf7ac9ae0af7d6b985905d384a899428d8bc721f011e91ff3b6b98: Status 404 returned error can't find the container with id f401434bbccf7ac9ae0af7d6b985905d384a899428d8bc721f011e91ff3b6b98
Jul 24 17:30:14.944572 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.944493    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92" event={"ID":"d191876f-0f3c-4526-bb8a-b8ced28e9dc1","Type":"ContainerStarted","Data":"94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd"}
Jul 24 17:30:14.944572 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.944531    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92" event={"ID":"d191876f-0f3c-4526-bb8a-b8ced28e9dc1","Type":"ContainerStarted","Data":"f401434bbccf7ac9ae0af7d6b985905d384a899428d8bc721f011e91ff3b6b98"}
Jul 24 17:30:14.963243 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:14.963195    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92" podStartSLOduration=0.963179603 podStartE2EDuration="963.179603ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:14.961531609 +0000 UTC m=+2821.721340643" watchObservedRunningTime="2026-07-24 17:30:14.963179603 +0000 UTC m=+2821.722988602"
Jul 24 17:30:24.689071 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:24.689034    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:24.689475 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:24.689083    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:24.701965 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:24.701926    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:25.004874 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:25.004793    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:30:59.370591 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:30:59.370557    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"]
Jul 24 17:31:00.220478 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:00.220444    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"]
Jul 24 17:31:13.966585 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:13.966555    2569 scope.go:117] "RemoveContainer" containerID="d708dd28516b3e57bc123a628a77061483bbeb28bb47132e83320f5ce47d5be3"
Jul 24 17:31:13.975274 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:13.975256    2569 scope.go:117] "RemoveContainer" containerID="e8cda349a858e846d63342b34f15ec7ea50cab278768c732062c92413c94443b"
Jul 24 17:31:14.389528 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.389494    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl" podUID="415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" containerName="main" containerID="cri-o://08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed" gracePeriod=45
Jul 24 17:31:14.637208 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.637175    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:31:14.734702 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tls-certs\") pod \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") "
Jul 24 17:31:14.734702 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734653    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-home\") pod \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") "
Jul 24 17:31:14.734702 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734675    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-model-cache\") pod \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") "
Jul 24 17:31:14.734987 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734717    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tmp-dir\") pod \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") "
Jul 24 17:31:14.734987 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734812    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xr79\" (UniqueName: \"kubernetes.io/projected/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-kube-api-access-2xr79\") pod \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") "
Jul 24 17:31:14.734987 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734856    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-dshm\") pod \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\" (UID: \"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b\") "
Jul 24 17:31:14.734987 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734930    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-home" (OuterVolumeSpecName: "home") pod "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" (UID: "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:14.735192 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734981    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-model-cache" (OuterVolumeSpecName: "model-cache") pod "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" (UID: "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:14.735192 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.734996    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" (UID: "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:14.735294 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.735206    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:14.735294 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.735220    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:14.735294 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.735233    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:14.736817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.736789    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" (UID: "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:31:14.736933 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.736865    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-dshm" (OuterVolumeSpecName: "dshm") pod "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" (UID: "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:14.736933 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.736912    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-kube-api-access-2xr79" (OuterVolumeSpecName: "kube-api-access-2xr79") pod "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" (UID: "415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b"). InnerVolumeSpecName "kube-api-access-2xr79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:31:14.836229 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.836195    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:14.836229 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.836222    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xr79\" (UniqueName: \"kubernetes.io/projected/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-kube-api-access-2xr79\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:14.836229 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:14.836232    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:15.239648 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.239612    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92" podUID="d191876f-0f3c-4526-bb8a-b8ced28e9dc1" containerName="main" containerID="cri-o://94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd" gracePeriod=45
Jul 24 17:31:15.240060 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.240003    2569 generic.go:356] "Generic (PLEG): container finished" podID="415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" containerID="08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed" exitCode=0
Jul 24 17:31:15.240060 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.240054    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"
Jul 24 17:31:15.240418 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.240107    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl" event={"ID":"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b","Type":"ContainerDied","Data":"08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed"}
Jul 24 17:31:15.240418 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.240154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl" event={"ID":"415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b","Type":"ContainerDied","Data":"334d08ed1d0e9fdfe9695e8be8d05667744d683d982f1d17e00a4c2acc6b8e24"}
Jul 24 17:31:15.240418 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.240178    2569 scope.go:117] "RemoveContainer" containerID="08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed"
Jul 24 17:31:15.250088 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.250073    2569 scope.go:117] "RemoveContainer" containerID="08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed"
Jul 24 17:31:15.250340 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:31:15.250314    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed\": container with ID starting with 08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed not found: ID does not exist" containerID="08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed"
Jul 24 17:31:15.250402 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.250349    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed"} err="failed to get container status \"08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed\": rpc error: code = NotFound desc = could not find container \"08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed\": container with ID starting with 08051f46545ffbe12caed14c3fa071adf1c074090ac5cc56aa102aaa59ed2eed not found: ID does not exist"
Jul 24 17:31:15.264342 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.264306    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"]
Jul 24 17:31:15.270990 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.270970    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-de68c7/sub-test-model-de68c7-kserve-7859cc9985-zxdkl"]
Jul 24 17:31:15.487970 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.487948    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:31:15.542626 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542552    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-model-cache\") pod \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") "
Jul 24 17:31:15.542626 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542593    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tls-certs\") pod \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") "
Jul 24 17:31:15.542626 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542620    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csnw6\" (UniqueName: \"kubernetes.io/projected/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-kube-api-access-csnw6\") pod \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") "
Jul 24 17:31:15.542850 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542752    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-home\") pod \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") "
Jul 24 17:31:15.542850 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542803    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tmp-dir\") pod \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") "
Jul 24 17:31:15.542947 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542865    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "d191876f-0f3c-4526-bb8a-b8ced28e9dc1" (UID: "d191876f-0f3c-4526-bb8a-b8ced28e9dc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:15.542947 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542900    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-dshm\") pod \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\" (UID: \"d191876f-0f3c-4526-bb8a-b8ced28e9dc1\") "
Jul 24 17:31:15.543051 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.542999    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-home" (OuterVolumeSpecName: "home") pod "d191876f-0f3c-4526-bb8a-b8ced28e9dc1" (UID: "d191876f-0f3c-4526-bb8a-b8ced28e9dc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:15.543125 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.543099    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d191876f-0f3c-4526-bb8a-b8ced28e9dc1" (UID: "d191876f-0f3c-4526-bb8a-b8ced28e9dc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:15.543241 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.543219    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:15.543241 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.543242    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:15.543416 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.543253    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:15.544709 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.544684    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d191876f-0f3c-4526-bb8a-b8ced28e9dc1" (UID: "d191876f-0f3c-4526-bb8a-b8ced28e9dc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:31:15.544855 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.544756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-kube-api-access-csnw6" (OuterVolumeSpecName: "kube-api-access-csnw6") pod "d191876f-0f3c-4526-bb8a-b8ced28e9dc1" (UID: "d191876f-0f3c-4526-bb8a-b8ced28e9dc1"). InnerVolumeSpecName "kube-api-access-csnw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:31:15.544855 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.544819    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-dshm" (OuterVolumeSpecName: "dshm") pod "d191876f-0f3c-4526-bb8a-b8ced28e9dc1" (UID: "d191876f-0f3c-4526-bb8a-b8ced28e9dc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:31:15.630171 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.630133    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" path="/var/lib/kubelet/pods/415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b/volumes"
Jul 24 17:31:15.644622 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.644598    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:15.644622 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.644621    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:15.644771 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:15.644633    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csnw6\" (UniqueName: \"kubernetes.io/projected/d191876f-0f3c-4526-bb8a-b8ced28e9dc1-kube-api-access-csnw6\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:31:16.245837 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.245790    2569 generic.go:356] "Generic (PLEG): container finished" podID="d191876f-0f3c-4526-bb8a-b8ced28e9dc1" containerID="94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd" exitCode=0
Jul 24 17:31:16.245837 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.245844    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"
Jul 24 17:31:16.246368 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.245878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92" event={"ID":"d191876f-0f3c-4526-bb8a-b8ced28e9dc1","Type":"ContainerDied","Data":"94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd"}
Jul 24 17:31:16.246368 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.245916    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92" event={"ID":"d191876f-0f3c-4526-bb8a-b8ced28e9dc1","Type":"ContainerDied","Data":"f401434bbccf7ac9ae0af7d6b985905d384a899428d8bc721f011e91ff3b6b98"}
Jul 24 17:31:16.246368 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.245933    2569 scope.go:117] "RemoveContainer" containerID="94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd"
Jul 24 17:31:16.254099 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.254084    2569 scope.go:117] "RemoveContainer" containerID="94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd"
Jul 24 17:31:16.254364 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:31:16.254342    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd\": container with ID starting with 94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd not found: ID does not exist" containerID="94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd"
Jul 24 17:31:16.254452 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.254375    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd"} err="failed to get container status \"94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd\": rpc error: code = NotFound desc = could not find container \"94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd\": container with ID starting with 94dd768d17825a951b565a1be1d46d253c533dc0eb59390e90fe070459a2a9bd not found: ID does not exist"
Jul 24 17:31:16.262321 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.262294    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"]
Jul 24 17:31:16.267834 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:16.267812    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/sub-test-model-d73e95-kserve-75cbf7df69-4sf92"]
Jul 24 17:31:17.630038 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:31:17.630003    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d191876f-0f3c-4526-bb8a-b8ced28e9dc1" path="/var/lib/kubelet/pods/d191876f-0f3c-4526-bb8a-b8ced28e9dc1/volumes"
Jul 24 17:32:13.366976 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.366937    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"]
Jul 24 17:32:13.369573 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.367462    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" containerName="main"
Jul 24 17:32:13.369573 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.367478    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" containerName="main"
Jul 24 17:32:13.369573 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.367489    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d191876f-0f3c-4526-bb8a-b8ced28e9dc1" containerName="main"
Jul 24 17:32:13.369573 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.367495    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d191876f-0f3c-4526-bb8a-b8ced28e9dc1" containerName="main"
Jul 24 17:32:13.369573 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.367565    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d191876f-0f3c-4526-bb8a-b8ced28e9dc1" containerName="main"
Jul 24 17:32:13.369573 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.367574    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="415e9c8e-92c3-4ce5-8ad1-c723dcc8c84b" containerName="main"
Jul 24 17:32:13.370494 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.370474    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.374658 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.374637    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"default-dockercfg-2f7c7\""
Jul 24 17:32:13.375691 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.375455    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"kube-root-ca.crt\""
Jul 24 17:32:13.375691 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.375504    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"rate-test-model-d73e95-kserve-self-signed-certs\""
Jul 24 17:32:13.375691 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.375523    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"openshift-service-ca.crt\""
Jul 24 17:32:13.399432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.399405    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"]
Jul 24 17:32:13.469905 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.469875    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvlt5\" (UniqueName: \"kubernetes.io/projected/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-kube-api-access-zvlt5\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.470104 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.469918    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tls-certs\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.470104 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.470006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-model-cache\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.470104 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.470056    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-home\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.470255 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.470125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tmp-dir\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.470255 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.470143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-dshm\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.570608 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.570577    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvlt5\" (UniqueName: \"kubernetes.io/projected/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-kube-api-access-zvlt5\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.570790 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.570618    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tls-certs\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.570790 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.570649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-model-cache\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.570790 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.570672    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-home\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.570790 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.570700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tmp-dir\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.570790 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.570724    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-dshm\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.571131 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.571081    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-model-cache\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.571229 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.571131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tmp-dir\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.571294 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.571241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-home\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.573081 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.573061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-dshm\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.575703 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.575688    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"rate-test-model-d73e95-kserve-self-signed-certs\""
Jul 24 17:32:13.583831 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.583809    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tls-certs\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.589666 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.589648    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"kube-root-ca.crt\""
Jul 24 17:32:13.601538 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.601515    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"openshift-service-ca.crt\""
Jul 24 17:32:13.609123 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.609097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvlt5\" (UniqueName: \"kubernetes.io/projected/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-kube-api-access-zvlt5\") pod \"rate-test-model-d73e95-kserve-5d6bfd59-jj574\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") " pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.683407 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.683299    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d73e95\"/\"default-dockercfg-2f7c7\""
Jul 24 17:32:13.691394 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.691374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:13.846564 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.846391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"]
Jul 24 17:32:13.849297 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:32:13.849269    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c0f05e6_4e17_41a3_a579_09a4dbd6eccc.slice/crio-809bde2adb627275a090cd80a1362b8e69ce97085f59e81c8bc85820cd2ccd38 WatchSource:0}: Error finding container 809bde2adb627275a090cd80a1362b8e69ce97085f59e81c8bc85820cd2ccd38: Status 404 returned error can't find the container with id 809bde2adb627275a090cd80a1362b8e69ce97085f59e81c8bc85820cd2ccd38
Jul 24 17:32:13.999020 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:13.998942    2569 scope.go:117] "RemoveContainer" containerID="2b1ecf1114494efcfebb06e059f437c66ce3e04c3e5fb4bfcb3c974a6fa6aecb"
Jul 24 17:32:14.522113 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:14.522074    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574" event={"ID":"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc","Type":"ContainerStarted","Data":"09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4"}
Jul 24 17:32:14.522548 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:14.522128    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574" event={"ID":"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc","Type":"ContainerStarted","Data":"809bde2adb627275a090cd80a1362b8e69ce97085f59e81c8bc85820cd2ccd38"}
Jul 24 17:32:23.691546 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:23.691504    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:23.692016 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:23.691562    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:23.704225 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:23.704200    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:32:23.722746 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:23.722689    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574" podStartSLOduration=10.722673751 podStartE2EDuration="10.722673751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:32:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:32:14.554429499 +0000 UTC m=+2941.314238498" watchObservedRunningTime="2026-07-24 17:32:23.722673751 +0000 UTC m=+2950.482482749"
Jul 24 17:32:24.585704 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:32:24.585669    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:33:00.003618 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:00.003524    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"]
Jul 24 17:33:06.863269 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:06.863234    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7508-139959428970512"]
Jul 24 17:33:06.866628 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:06.866610    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:06.872679 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:06.872656    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-24fmh\""
Jul 24 17:33:06.875770 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:06.875744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7508-139959428970512"]
Jul 24 17:33:07.034744 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:07.034708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkv44\" (UniqueName: \"kubernetes.io/projected/4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345-kube-api-access-hkv44\") pod \"test-curl-7508-139959428970512\" (UID: \"4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345\") " pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:07.136025 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:07.135934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkv44\" (UniqueName: \"kubernetes.io/projected/4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345-kube-api-access-hkv44\") pod \"test-curl-7508-139959428970512\" (UID: \"4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345\") " pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:07.144969 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:07.144948    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkv44\" (UniqueName: \"kubernetes.io/projected/4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345-kube-api-access-hkv44\") pod \"test-curl-7508-139959428970512\" (UID: \"4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345\") " pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:07.176844 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:07.176803    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:07.529636 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:07.529613    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7508-139959428970512"]
Jul 24 17:33:07.531173 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:33:07.531141    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f8d3b5c_ac5c_4df0_a5dc_9eed0cbf4345.slice/crio-7e3b2d7a3a3176e339059b23d4134dade893cfa6eb762567cb06048e29a06bce WatchSource:0}: Error finding container 7e3b2d7a3a3176e339059b23d4134dade893cfa6eb762567cb06048e29a06bce: Status 404 returned error can't find the container with id 7e3b2d7a3a3176e339059b23d4134dade893cfa6eb762567cb06048e29a06bce
Jul 24 17:33:07.532588 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:07.532569    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:33:07.772678 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:07.772646    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7508-139959428970512" event={"ID":"4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345","Type":"ContainerStarted","Data":"7e3b2d7a3a3176e339059b23d4134dade893cfa6eb762567cb06048e29a06bce"}
Jul 24 17:33:08.779408 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:08.779372    2569 generic.go:356] "Generic (PLEG): container finished" podID="4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345" containerID="2c5c2314e69f70830f5a9b29b8a5ea1ebfc8fe548cd988c5ed1ed008d638037b" exitCode=0
Jul 24 17:33:08.779408 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:08.779410    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7508-139959428970512" event={"ID":"4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345","Type":"ContainerDied","Data":"2c5c2314e69f70830f5a9b29b8a5ea1ebfc8fe548cd988c5ed1ed008d638037b"}
Jul 24 17:33:09.913553 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:09.913529    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:09.949581 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:09.949555    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7508-139959428970512_4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345/test-curl-7508-139959428970512/0.log"
Jul 24 17:33:09.974155 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:09.974127    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7508-139959428970512"]
Jul 24 17:33:09.978504 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:09.978472    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7508-139959428970512"]
Jul 24 17:33:10.063531 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:10.063454    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkv44\" (UniqueName: \"kubernetes.io/projected/4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345-kube-api-access-hkv44\") pod \"4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345\" (UID: \"4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345\") "
Jul 24 17:33:10.065698 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:10.065662    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345-kube-api-access-hkv44" (OuterVolumeSpecName: "kube-api-access-hkv44") pod "4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345" (UID: "4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345"). InnerVolumeSpecName "kube-api-access-hkv44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:33:10.164939 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:10.164901    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkv44\" (UniqueName: \"kubernetes.io/projected/4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345-kube-api-access-hkv44\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:10.790621 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:10.790588    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e3b2d7a3a3176e339059b23d4134dade893cfa6eb762567cb06048e29a06bce"
Jul 24 17:33:10.790621 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:10.790602    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:10.800742 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:33:10.800711    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7508-139959428970512\" is forbidden: User \"system:node:ip-10-0-135-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-204.ec2.internal' and this object" podUID="4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345" pod="opendatahub/test-curl-7508-139959428970512"
Jul 24 17:33:11.636044 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:11.636001    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345" path="/var/lib/kubelet/pods/4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345/volumes"
Jul 24 17:33:15.023741 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.023692    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574" podUID="6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" containerName="main" containerID="cri-o://09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4" gracePeriod=45
Jul 24 17:33:15.278063 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.278002    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:33:15.412911 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.412880    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-dshm\") pod \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") "
Jul 24 17:33:15.413114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.412933    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-model-cache\") pod \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") "
Jul 24 17:33:15.413114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.412965    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tls-certs\") pod \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") "
Jul 24 17:33:15.413114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.412999    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-home\") pod \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") "
Jul 24 17:33:15.413114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413038    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tmp-dir\") pod \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") "
Jul 24 17:33:15.413352 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413206    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvlt5\" (UniqueName: \"kubernetes.io/projected/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-kube-api-access-zvlt5\") pod \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\" (UID: \"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc\") "
Jul 24 17:33:15.413352 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413272    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" (UID: "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:15.413352 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413290    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-home" (OuterVolumeSpecName: "home") pod "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" (UID: "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:15.413352 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413299    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" (UID: "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:15.413539 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413523    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:15.413577 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413542    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:15.413577 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.413552    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:15.415201 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.415167    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-dshm" (OuterVolumeSpecName: "dshm") pod "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" (UID: "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:15.415201 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.415182    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-kube-api-access-zvlt5" (OuterVolumeSpecName: "kube-api-access-zvlt5") pod "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" (UID: "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc"). InnerVolumeSpecName "kube-api-access-zvlt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:33:15.415343 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.415200    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" (UID: "6c0f05e6-4e17-41a3-a579-09a4dbd6eccc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:33:15.514745 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.514710    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvlt5\" (UniqueName: \"kubernetes.io/projected/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-kube-api-access-zvlt5\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:15.514745 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.514741    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:15.514745 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.514751    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:15.814438 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.814406    2569 generic.go:356] "Generic (PLEG): container finished" podID="6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" containerID="09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4" exitCode=0
Jul 24 17:33:15.814637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.814449    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"
Jul 24 17:33:15.814637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.814501    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574" event={"ID":"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc","Type":"ContainerDied","Data":"09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4"}
Jul 24 17:33:15.814637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.814552    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574" event={"ID":"6c0f05e6-4e17-41a3-a579-09a4dbd6eccc","Type":"ContainerDied","Data":"809bde2adb627275a090cd80a1362b8e69ce97085f59e81c8bc85820cd2ccd38"}
Jul 24 17:33:15.814637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.814574    2569 scope.go:117] "RemoveContainer" containerID="09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4"
Jul 24 17:33:15.823508 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.823478    2569 scope.go:117] "RemoveContainer" containerID="09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4"
Jul 24 17:33:15.823777 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:33:15.823746    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4\": container with ID starting with 09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4 not found: ID does not exist" containerID="09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4"
Jul 24 17:33:15.823851 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.823788    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4"} err="failed to get container status \"09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4\": rpc error: code = NotFound desc = could not find container \"09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4\": container with ID starting with 09378098eb062348e39c52293e1c186b15c92e16a09198cd23bbc45f3f249aa4 not found: ID does not exist"
Jul 24 17:33:15.830850 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.830809    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"]
Jul 24 17:33:15.832535 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.832517    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d73e95/rate-test-model-d73e95-kserve-5d6bfd59-jj574"]
Jul 24 17:33:15.856223 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.856195    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7508-139959434282448"]
Jul 24 17:33:15.856548 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.856535    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" containerName="main"
Jul 24 17:33:15.856590 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.856549    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" containerName="main"
Jul 24 17:33:15.856590 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.856563    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345" containerName="test-curl-7508-139959428970512"
Jul 24 17:33:15.856590 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.856569    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345" containerName="test-curl-7508-139959428970512"
Jul 24 17:33:15.856695 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.856632    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" containerName="main"
Jul 24 17:33:15.856695 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.856642    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f8d3b5c-ac5c-4df0-a5dc-9eed0cbf4345" containerName="test-curl-7508-139959428970512"
Jul 24 17:33:15.860831 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.860815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959434282448"
Jul 24 17:33:15.864185 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.864167    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-24fmh\""
Jul 24 17:33:15.879513 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:15.879490    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7508-139959434282448"]
Jul 24 17:33:16.018758 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:16.018665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vz5p\" (UniqueName: \"kubernetes.io/projected/c98b6521-f19b-4468-8e08-25dcea02fcd3-kube-api-access-4vz5p\") pod \"test-curl-7508-139959434282448\" (UID: \"c98b6521-f19b-4468-8e08-25dcea02fcd3\") " pod="opendatahub/test-curl-7508-139959434282448"
Jul 24 17:33:16.120001 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:16.119963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vz5p\" (UniqueName: \"kubernetes.io/projected/c98b6521-f19b-4468-8e08-25dcea02fcd3-kube-api-access-4vz5p\") pod \"test-curl-7508-139959434282448\" (UID: \"c98b6521-f19b-4468-8e08-25dcea02fcd3\") " pod="opendatahub/test-curl-7508-139959434282448"
Jul 24 17:33:16.129843 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:16.129805    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vz5p\" (UniqueName: \"kubernetes.io/projected/c98b6521-f19b-4468-8e08-25dcea02fcd3-kube-api-access-4vz5p\") pod \"test-curl-7508-139959434282448\" (UID: \"c98b6521-f19b-4468-8e08-25dcea02fcd3\") " pod="opendatahub/test-curl-7508-139959434282448"
Jul 24 17:33:16.169651 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:16.169619    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959434282448"
Jul 24 17:33:16.508767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:16.508745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7508-139959434282448"]
Jul 24 17:33:16.511264 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:33:16.511229    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc98b6521_f19b_4468_8e08_25dcea02fcd3.slice/crio-61a5b13a216a08d392f24b13a36add723d8b1f3c90bf7471352ebcdaaa5ec211 WatchSource:0}: Error finding container 61a5b13a216a08d392f24b13a36add723d8b1f3c90bf7471352ebcdaaa5ec211: Status 404 returned error can't find the container with id 61a5b13a216a08d392f24b13a36add723d8b1f3c90bf7471352ebcdaaa5ec211
Jul 24 17:33:16.821174 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:16.821145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7508-139959434282448" event={"ID":"c98b6521-f19b-4468-8e08-25dcea02fcd3","Type":"ContainerStarted","Data":"61a5b13a216a08d392f24b13a36add723d8b1f3c90bf7471352ebcdaaa5ec211"}
Jul 24 17:33:17.630443 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:17.630405    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c0f05e6-4e17-41a3-a579-09a4dbd6eccc" path="/var/lib/kubelet/pods/6c0f05e6-4e17-41a3-a579-09a4dbd6eccc/volumes"
Jul 24 17:33:17.828782 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:17.828746    2569 generic.go:356] "Generic (PLEG): container finished" podID="c98b6521-f19b-4468-8e08-25dcea02fcd3" containerID="6bcac8ad0ad305e6510020ef95d1f2a90520ded7c11c728804cee8ec71965d29" exitCode=0
Jul 24 17:33:17.828950 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:17.828809    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7508-139959434282448" event={"ID":"c98b6521-f19b-4468-8e08-25dcea02fcd3","Type":"ContainerDied","Data":"6bcac8ad0ad305e6510020ef95d1f2a90520ded7c11c728804cee8ec71965d29"}
Jul 24 17:33:18.959980 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:18.959954    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959434282448"
Jul 24 17:33:18.983693 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:18.983670    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7508-139959434282448_c98b6521-f19b-4468-8e08-25dcea02fcd3/test-curl-7508-139959434282448/0.log"
Jul 24 17:33:19.016636 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.016583    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7508-139959434282448"]
Jul 24 17:33:19.018425 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.018403    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7508-139959434282448"]
Jul 24 17:33:19.148252 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.148181    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vz5p\" (UniqueName: \"kubernetes.io/projected/c98b6521-f19b-4468-8e08-25dcea02fcd3-kube-api-access-4vz5p\") pod \"c98b6521-f19b-4468-8e08-25dcea02fcd3\" (UID: \"c98b6521-f19b-4468-8e08-25dcea02fcd3\") "
Jul 24 17:33:19.150323 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.150300    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98b6521-f19b-4468-8e08-25dcea02fcd3-kube-api-access-4vz5p" (OuterVolumeSpecName: "kube-api-access-4vz5p") pod "c98b6521-f19b-4468-8e08-25dcea02fcd3" (UID: "c98b6521-f19b-4468-8e08-25dcea02fcd3"). InnerVolumeSpecName "kube-api-access-4vz5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:33:19.249100 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.249070    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vz5p\" (UniqueName: \"kubernetes.io/projected/c98b6521-f19b-4468-8e08-25dcea02fcd3-kube-api-access-4vz5p\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:33:19.629933 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.629901    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c98b6521-f19b-4468-8e08-25dcea02fcd3" path="/var/lib/kubelet/pods/c98b6521-f19b-4468-8e08-25dcea02fcd3/volumes"
Jul 24 17:33:19.839050 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.839023    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7508-139959434282448"
Jul 24 17:33:19.839050 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:19.839034    2569 scope.go:117] "RemoveContainer" containerID="6bcac8ad0ad305e6510020ef95d1f2a90520ded7c11c728804cee8ec71965d29"
Jul 24 17:33:22.831658 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.831621    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"]
Jul 24 17:33:22.832161 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.832144    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c98b6521-f19b-4468-8e08-25dcea02fcd3" containerName="test-curl-7508-139959434282448"
Jul 24 17:33:22.832220 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.832164    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98b6521-f19b-4468-8e08-25dcea02fcd3" containerName="test-curl-7508-139959434282448"
Jul 24 17:33:22.832266 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.832256    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c98b6521-f19b-4468-8e08-25dcea02fcd3" containerName="test-curl-7508-139959434282448"
Jul 24 17:33:22.836890 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.836871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.840521 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.840499    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-6a1c15-serving-cert\""
Jul 24 17:33:22.842674 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.842651    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hqnrd\""
Jul 24 17:33:22.842786 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.842697    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:33:22.846157 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.846138    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:33:22.857078 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.857057    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"]
Jul 24 17:33:22.879284 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.879256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmlvz\" (UniqueName: \"kubernetes.io/projected/e1035f27-51a3-4225-809a-625174c526de-kube-api-access-dmlvz\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.879430 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.879291    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1035f27-51a3-4225-809a-625174c526de-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.879430 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.879345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.980718 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.980686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1035f27-51a3-4225-809a-625174c526de-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.980893 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.980731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.980893 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.980795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmlvz\" (UniqueName: \"kubernetes.io/projected/e1035f27-51a3-4225-809a-625174c526de-kube-api-access-dmlvz\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.981013 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:33:22.980912    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-serving-cert: secret "maas-api-e2e-ipp-a-6a1c15-serving-cert" not found
Jul 24 17:33:22.981013 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:33:22.980989    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls podName:e1035f27-51a3-4225-809a-625174c526de nodeName:}" failed. No retries permitted until 2026-07-24 17:33:23.480966526 +0000 UTC m=+3010.240775503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls") pod "maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st" (UID: "e1035f27-51a3-4225-809a-625174c526de") : secret "maas-api-e2e-ipp-a-6a1c15-serving-cert" not found
Jul 24 17:33:22.981368 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.981320    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1035f27-51a3-4225-809a-625174c526de-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:22.989682 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:22.989659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmlvz\" (UniqueName: \"kubernetes.io/projected/e1035f27-51a3-4225-809a-625174c526de-kube-api-access-dmlvz\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:23.484963 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:23.484931    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:23.487279 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:23.487250    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:23.747628 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:23.747554    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:23.891173 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:23.891147    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"]
Jul 24 17:33:23.893287 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:33:23.893259    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1035f27_51a3_4225_809a_625174c526de.slice/crio-dcc42942bd1ffa5c58287cfcfabaa779e92098f2a2d62a3ad4ec02e0ec5129e8 WatchSource:0}: Error finding container dcc42942bd1ffa5c58287cfcfabaa779e92098f2a2d62a3ad4ec02e0ec5129e8: Status 404 returned error can't find the container with id dcc42942bd1ffa5c58287cfcfabaa779e92098f2a2d62a3ad4ec02e0ec5129e8
Jul 24 17:33:24.871423 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:24.871323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st" event={"ID":"e1035f27-51a3-4225-809a-625174c526de","Type":"ContainerStarted","Data":"dcc42942bd1ffa5c58287cfcfabaa779e92098f2a2d62a3ad4ec02e0ec5129e8"}
Jul 24 17:33:26.881600 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:26.881564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st" event={"ID":"e1035f27-51a3-4225-809a-625174c526de","Type":"ContainerStarted","Data":"b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a"}
Jul 24 17:33:26.882226 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:26.881657    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:26.900695 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:26.900642    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st" podStartSLOduration=2.531415647 podStartE2EDuration="4.900627096s" totalImagesPullingTime="2.369211449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:33:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:33:26.899282865 +0000 UTC m=+3013.659091865" watchObservedRunningTime="2026-07-24 17:33:26.900627096 +0000 UTC m=+3013.660436091"
Jul 24 17:33:32.892169 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:32.892140    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:33:41.945637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:41.945602    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"]
Jul 24 17:33:41.952114 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:41.952093    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:41.954697 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:41.954675    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a2f844-serving-cert\""
Jul 24 17:33:41.963214 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:41.963187    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"]
Jul 24 17:33:42.054596 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.054560    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.054775 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.054630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.054775 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.054654    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnzfz\" (UniqueName: \"kubernetes.io/projected/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-kube-api-access-pnzfz\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.155764 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.155731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.155978 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.155776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnzfz\" (UniqueName: \"kubernetes.io/projected/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-kube-api-access-pnzfz\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.155978 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.155822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.156515 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.156490    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.158210 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.158189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.169396 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.169366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnzfz\" (UniqueName: \"kubernetes.io/projected/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-kube-api-access-pnzfz\") pod \"maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.265153 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.265069    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.404614 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.404582    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"]
Jul 24 17:33:42.405821 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:33:42.405794    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30f5b34_adee_467a_b6d0_ff2c8c0a72c1.slice/crio-5d286c370e337b8d388fda2b92930c0dae1ed836b8e0671120248157afe56c88 WatchSource:0}: Error finding container 5d286c370e337b8d388fda2b92930c0dae1ed836b8e0671120248157afe56c88: Status 404 returned error can't find the container with id 5d286c370e337b8d388fda2b92930c0dae1ed836b8e0671120248157afe56c88
Jul 24 17:33:42.963671 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.963636    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr" event={"ID":"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1","Type":"ContainerStarted","Data":"4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1"}
Jul 24 17:33:42.964042 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.963688    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr" event={"ID":"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1","Type":"ContainerStarted","Data":"5d286c370e337b8d388fda2b92930c0dae1ed836b8e0671120248157afe56c88"}
Jul 24 17:33:42.964042 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.963731    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:33:42.991217 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:42.991165    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr" podStartSLOduration=1.637062504 podStartE2EDuration="1.991149328s" totalImagesPullingTime="354.086824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:33:42.990850006 +0000 UTC m=+3029.750659006" watchObservedRunningTime="2026-07-24 17:33:42.991149328 +0000 UTC m=+3029.750958329"
Jul 24 17:33:48.974469 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:33:48.974438    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:34:12.390986 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.390912    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"]
Jul 24 17:34:12.398742 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.398719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.401459 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.401433    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6a1c15\"/\"default-dockercfg-d7b2f\""
Jul 24 17:34:12.401459 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.401434    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6a1c15\"/\"openshift-service-ca.crt\""
Jul 24 17:34:12.401786 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.401434    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6a1c15\"/\"kube-root-ca.crt\""
Jul 24 17:34:12.402644 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.402621    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6a1c15\"/\"ipp-route-6a1c15-kserve-self-signed-certs\""
Jul 24 17:34:12.405285 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.405264    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"]
Jul 24 17:34:12.520593 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.520547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-tmp-dir\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.520593 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.520606    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-home\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.520879 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.520676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-dshm\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.520879 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.520706    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-model-cache\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.520879 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.520723    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqgq4\" (UniqueName: \"kubernetes.io/projected/9efc6b5f-6900-4119-8b4f-284c445be4bf-kube-api-access-mqgq4\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.520879 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.520747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efc6b5f-6900-4119-8b4f-284c445be4bf-tls-certs\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.621621 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.621583    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-dshm\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.621919 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.621638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-model-cache\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.621919 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.621658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqgq4\" (UniqueName: \"kubernetes.io/projected/9efc6b5f-6900-4119-8b4f-284c445be4bf-kube-api-access-mqgq4\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.621919 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.621685    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efc6b5f-6900-4119-8b4f-284c445be4bf-tls-certs\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.621919 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.621712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-tmp-dir\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.621919 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.621739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-home\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.622183 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.622055    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-model-cache\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.622183 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.622109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-home\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.622291 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.622218    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-tmp-dir\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.623972 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.623945    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-dshm\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.624268 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.624251    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efc6b5f-6900-4119-8b4f-284c445be4bf-tls-certs\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.632073 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.632050    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqgq4\" (UniqueName: \"kubernetes.io/projected/9efc6b5f-6900-4119-8b4f-284c445be4bf-kube-api-access-mqgq4\") pod \"ipp-route-6a1c15-kserve-79bbc78959-rh5v4\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") " pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.710098 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.710004    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:12.841964 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:12.841789    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"]
Jul 24 17:34:12.844442 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:34:12.844416    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9efc6b5f_6900_4119_8b4f_284c445be4bf.slice/crio-802ccfec9ade2e691fa9ac770e5319badf11611aae3399d2161da39f0ebc1d87 WatchSource:0}: Error finding container 802ccfec9ade2e691fa9ac770e5319badf11611aae3399d2161da39f0ebc1d87: Status 404 returned error can't find the container with id 802ccfec9ade2e691fa9ac770e5319badf11611aae3399d2161da39f0ebc1d87
Jul 24 17:34:13.113229 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:13.113188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4" event={"ID":"9efc6b5f-6900-4119-8b4f-284c445be4bf","Type":"ContainerStarted","Data":"e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b"}
Jul 24 17:34:13.113470 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:13.113239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4" event={"ID":"9efc6b5f-6900-4119-8b4f-284c445be4bf","Type":"ContainerStarted","Data":"802ccfec9ade2e691fa9ac770e5319badf11611aae3399d2161da39f0ebc1d87"}
Jul 24 17:34:13.130006 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:13.129953    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4" podStartSLOduration=1.129938746 podStartE2EDuration="1.129938746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:34:13.129291061 +0000 UTC m=+3059.889100062" watchObservedRunningTime="2026-07-24 17:34:13.129938746 +0000 UTC m=+3059.889747744"
Jul 24 17:34:22.710475 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:22.710426    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:22.710955 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:22.710707    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:22.723653 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:22.723631    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:23.174242 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:23.174216    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:34:49.525968 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.525927    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"]
Jul 24 17:34:49.529799 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.529777    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.532756 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.532730    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-600cad-serving-cert\""
Jul 24 17:34:49.541912 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.541885    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"]
Jul 24 17:34:49.668855 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.668821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff630d64-f237-4f06-b35a-3c6bc25c1497-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.668855 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.668857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff630d64-f237-4f06-b35a-3c6bc25c1497-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.669067 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.668975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjv66\" (UniqueName: \"kubernetes.io/projected/ff630d64-f237-4f06-b35a-3c6bc25c1497-kube-api-access-zjv66\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.769877 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.769842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjv66\" (UniqueName: \"kubernetes.io/projected/ff630d64-f237-4f06-b35a-3c6bc25c1497-kube-api-access-zjv66\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.770155 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.770106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff630d64-f237-4f06-b35a-3c6bc25c1497-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.770155 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.770149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff630d64-f237-4f06-b35a-3c6bc25c1497-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.770785 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.770762    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff630d64-f237-4f06-b35a-3c6bc25c1497-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.772565 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.772543    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff630d64-f237-4f06-b35a-3c6bc25c1497-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.779057 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.778995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjv66\" (UniqueName: \"kubernetes.io/projected/ff630d64-f237-4f06-b35a-3c6bc25c1497-kube-api-access-zjv66\") pod \"maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.840925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.840888    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:49.979190 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:49.979161    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"]
Jul 24 17:34:49.981101 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:34:49.981075    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff630d64_f237_4f06_b35a_3c6bc25c1497.slice/crio-b099903aba5ed57d2ef548e60076e4e5ef8ddb2a4db1848298f1ce3967771e1c WatchSource:0}: Error finding container b099903aba5ed57d2ef548e60076e4e5ef8ddb2a4db1848298f1ce3967771e1c: Status 404 returned error can't find the container with id b099903aba5ed57d2ef548e60076e4e5ef8ddb2a4db1848298f1ce3967771e1c
Jul 24 17:34:50.293686 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:50.293648    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6" event={"ID":"ff630d64-f237-4f06-b35a-3c6bc25c1497","Type":"ContainerStarted","Data":"b099903aba5ed57d2ef548e60076e4e5ef8ddb2a4db1848298f1ce3967771e1c"}
Jul 24 17:34:51.300964 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:51.300921    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6" event={"ID":"ff630d64-f237-4f06-b35a-3c6bc25c1497","Type":"ContainerStarted","Data":"3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e"}
Jul 24 17:34:51.301479 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:51.301082    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:34:51.318424 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:51.318369    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6" podStartSLOduration=1.935740322 podStartE2EDuration="2.318319968s" totalImagesPullingTime="382.579646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:34:51.318098958 +0000 UTC m=+3098.077907960" watchObservedRunningTime="2026-07-24 17:34:51.318319968 +0000 UTC m=+3098.078128966"
Jul 24 17:34:57.312306 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:34:57.312275    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:35:03.482808 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.482770    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"]
Jul 24 17:35:03.486382 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.486364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"
Jul 24 17:35:03.488675 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.488653    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:35:03.493303 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.493274    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"]
Jul 24 17:35:03.583066 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.583027    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f170b0e3-ba3f-4b69-b2e1-d19e2648cd03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l\" (UID: \"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"
Jul 24 17:35:03.684106 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.684062    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f170b0e3-ba3f-4b69-b2e1-d19e2648cd03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l\" (UID: \"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"
Jul 24 17:35:03.684723 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.684704    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f170b0e3-ba3f-4b69-b2e1-d19e2648cd03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l\" (UID: \"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"
Jul 24 17:35:03.798904 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.798817    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"
Jul 24 17:35:03.932853 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:03.932801    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"]
Jul 24 17:35:03.935628 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:35:03.935599    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf170b0e3_ba3f_4b69_b2e1_d19e2648cd03.slice/crio-6985c0f8f2997f35d0ac89048b7b29fc0a5b35d2aaf4ecdebc4fca6c8b9a447a WatchSource:0}: Error finding container 6985c0f8f2997f35d0ac89048b7b29fc0a5b35d2aaf4ecdebc4fca6c8b9a447a: Status 404 returned error can't find the container with id 6985c0f8f2997f35d0ac89048b7b29fc0a5b35d2aaf4ecdebc4fca6c8b9a447a
Jul 24 17:35:04.367022 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:04.366985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l" event={"ID":"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03","Type":"ContainerStarted","Data":"fc055543d63d18bb8f2909f7f4040110c3074afd3fa1a465c73f348cbe38b9a5"}
Jul 24 17:35:04.367226 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:04.367048    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l" event={"ID":"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03","Type":"ContainerStarted","Data":"6985c0f8f2997f35d0ac89048b7b29fc0a5b35d2aaf4ecdebc4fca6c8b9a447a"}
Jul 24 17:35:04.382757 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:04.382707    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l" podStartSLOduration=1.382690862 podStartE2EDuration="1.382690862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:35:04.381196974 +0000 UTC m=+3111.141005968" watchObservedRunningTime="2026-07-24 17:35:04.382690862 +0000 UTC m=+3111.142499860"
Jul 24 17:35:05.373239 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:05.373198    2569 generic.go:356] "Generic (PLEG): container finished" podID="f170b0e3-ba3f-4b69-b2e1-d19e2648cd03" containerID="fc055543d63d18bb8f2909f7f4040110c3074afd3fa1a465c73f348cbe38b9a5" exitCode=0
Jul 24 17:35:05.373645 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:05.373284    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l" event={"ID":"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03","Type":"ContainerDied","Data":"fc055543d63d18bb8f2909f7f4040110c3074afd3fa1a465c73f348cbe38b9a5"}
Jul 24 17:35:06.508493 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:06.508468    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"
Jul 24 17:35:06.612724 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:06.612683    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f170b0e3-ba3f-4b69-b2e1-d19e2648cd03-maas-api-service-ca\") pod \"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03\" (UID: \"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03\") "
Jul 24 17:35:06.613052 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:06.613029    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f170b0e3-ba3f-4b69-b2e1-d19e2648cd03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f170b0e3-ba3f-4b69-b2e1-d19e2648cd03" (UID: "f170b0e3-ba3f-4b69-b2e1-d19e2648cd03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:35:06.714410 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:06.714289    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f170b0e3-ba3f-4b69-b2e1-d19e2648cd03-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:07.384674 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:07.384642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l" event={"ID":"f170b0e3-ba3f-4b69-b2e1-d19e2648cd03","Type":"ContainerDied","Data":"6985c0f8f2997f35d0ac89048b7b29fc0a5b35d2aaf4ecdebc4fca6c8b9a447a"}
Jul 24 17:35:07.384859 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:07.384691    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6985c0f8f2997f35d0ac89048b7b29fc0a5b35d2aaf4ecdebc4fca6c8b9a447a"
Jul 24 17:35:07.384859 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:07.384657    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"
Jul 24 17:35:15.540071 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.540040    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"]
Jul 24 17:35:15.540569 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.540303    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6" podUID="ff630d64-f237-4f06-b35a-3c6bc25c1497" containerName="maas-api" containerID="cri-o://3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e" gracePeriod=30
Jul 24 17:35:15.790284 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.790226    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:35:15.898093 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.898064    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjv66\" (UniqueName: \"kubernetes.io/projected/ff630d64-f237-4f06-b35a-3c6bc25c1497-kube-api-access-zjv66\") pod \"ff630d64-f237-4f06-b35a-3c6bc25c1497\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") "
Jul 24 17:35:15.898257 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.898103    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff630d64-f237-4f06-b35a-3c6bc25c1497-openshift-service-ca\") pod \"ff630d64-f237-4f06-b35a-3c6bc25c1497\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") "
Jul 24 17:35:15.898257 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.898135    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff630d64-f237-4f06-b35a-3c6bc25c1497-maas-api-tls\") pod \"ff630d64-f237-4f06-b35a-3c6bc25c1497\" (UID: \"ff630d64-f237-4f06-b35a-3c6bc25c1497\") "
Jul 24 17:35:15.898578 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.898552    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff630d64-f237-4f06-b35a-3c6bc25c1497-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff630d64-f237-4f06-b35a-3c6bc25c1497" (UID: "ff630d64-f237-4f06-b35a-3c6bc25c1497"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:35:15.900445 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.900419    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff630d64-f237-4f06-b35a-3c6bc25c1497-kube-api-access-zjv66" (OuterVolumeSpecName: "kube-api-access-zjv66") pod "ff630d64-f237-4f06-b35a-3c6bc25c1497" (UID: "ff630d64-f237-4f06-b35a-3c6bc25c1497"). InnerVolumeSpecName "kube-api-access-zjv66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:35:15.900531 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.900423    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff630d64-f237-4f06-b35a-3c6bc25c1497-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff630d64-f237-4f06-b35a-3c6bc25c1497" (UID: "ff630d64-f237-4f06-b35a-3c6bc25c1497"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:35:15.999158 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.999118    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjv66\" (UniqueName: \"kubernetes.io/projected/ff630d64-f237-4f06-b35a-3c6bc25c1497-kube-api-access-zjv66\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:15.999158 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.999149    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff630d64-f237-4f06-b35a-3c6bc25c1497-openshift-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:15.999158 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:15.999159    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff630d64-f237-4f06-b35a-3c6bc25c1497-maas-api-tls\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:16.427985 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.427948    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff630d64-f237-4f06-b35a-3c6bc25c1497" containerID="3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e" exitCode=0
Jul 24 17:35:16.428186 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.428011    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"
Jul 24 17:35:16.428186 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.428035    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6" event={"ID":"ff630d64-f237-4f06-b35a-3c6bc25c1497","Type":"ContainerDied","Data":"3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e"}
Jul 24 17:35:16.428186 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.428123    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6" event={"ID":"ff630d64-f237-4f06-b35a-3c6bc25c1497","Type":"ContainerDied","Data":"b099903aba5ed57d2ef548e60076e4e5ef8ddb2a4db1848298f1ce3967771e1c"}
Jul 24 17:35:16.428186 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.428153    2569 scope.go:117] "RemoveContainer" containerID="3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e"
Jul 24 17:35:16.438151 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.438135    2569 scope.go:117] "RemoveContainer" containerID="3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e"
Jul 24 17:35:16.438433 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:35:16.438417    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e\": container with ID starting with 3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e not found: ID does not exist" containerID="3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e"
Jul 24 17:35:16.438498 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.438440    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e"} err="failed to get container status \"3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e\": rpc error: code = NotFound desc = could not find container \"3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e\": container with ID starting with 3e5387a04b79476c58fb2d14b684c7af1409b22d68ab2a98b886f9b1ba06415e not found: ID does not exist"
Jul 24 17:35:16.450231 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.450212    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"]
Jul 24 17:35:16.455536 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:16.455516    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-600cad-7bb979dc4c-pgvx6"]
Jul 24 17:35:17.630592 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:17.630556    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff630d64-f237-4f06-b35a-3c6bc25c1497" path="/var/lib/kubelet/pods/ff630d64-f237-4f06-b35a-3c6bc25c1497/volumes"
Jul 24 17:35:18.591248 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:18.591212    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"]
Jul 24 17:35:18.596820 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:18.596793    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-600cad-kgr5l"]
Jul 24 17:35:19.630171 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:19.630140    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f170b0e3-ba3f-4b69-b2e1-d19e2648cd03" path="/var/lib/kubelet/pods/f170b0e3-ba3f-4b69-b2e1-d19e2648cd03/volumes"
Jul 24 17:35:29.185748 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.185714    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"]
Jul 24 17:35:29.186236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.186062    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f170b0e3-ba3f-4b69-b2e1-d19e2648cd03" containerName="revoke-keys"
Jul 24 17:35:29.186236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.186074    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f170b0e3-ba3f-4b69-b2e1-d19e2648cd03" containerName="revoke-keys"
Jul 24 17:35:29.186236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.186089    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff630d64-f237-4f06-b35a-3c6bc25c1497" containerName="maas-api"
Jul 24 17:35:29.186236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.186096    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff630d64-f237-4f06-b35a-3c6bc25c1497" containerName="maas-api"
Jul 24 17:35:29.186236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.186161    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff630d64-f237-4f06-b35a-3c6bc25c1497" containerName="maas-api"
Jul 24 17:35:29.186236 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.186171    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f170b0e3-ba3f-4b69-b2e1-d19e2648cd03" containerName="revoke-keys"
Jul 24 17:35:29.189372 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.189351    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"
Jul 24 17:35:29.191441 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.191417    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:35:29.197508 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.197480    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"]
Jul 24 17:35:29.317031 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.316991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff8c3642-67d5-41ce-ac9c-395053f45ab6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht\" (UID: \"ff8c3642-67d5-41ce-ac9c-395053f45ab6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"
Jul 24 17:35:29.417584 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.417544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff8c3642-67d5-41ce-ac9c-395053f45ab6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht\" (UID: \"ff8c3642-67d5-41ce-ac9c-395053f45ab6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"
Jul 24 17:35:29.418270 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.418249    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff8c3642-67d5-41ce-ac9c-395053f45ab6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht\" (UID: \"ff8c3642-67d5-41ce-ac9c-395053f45ab6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"
Jul 24 17:35:29.501678 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.501592    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"
Jul 24 17:35:29.634391 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:29.634244    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"]
Jul 24 17:35:29.636762 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:35:29.636734    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8c3642_67d5_41ce_ac9c_395053f45ab6.slice/crio-6d2f9c5def860e3f5aab40d693fd9f88a735e9f59945c6ed79b0ce9672192332 WatchSource:0}: Error finding container 6d2f9c5def860e3f5aab40d693fd9f88a735e9f59945c6ed79b0ce9672192332: Status 404 returned error can't find the container with id 6d2f9c5def860e3f5aab40d693fd9f88a735e9f59945c6ed79b0ce9672192332
Jul 24 17:35:30.505748 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:30.505642    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff8c3642-67d5-41ce-ac9c-395053f45ab6" containerID="8d38c1c9ca609bcbcca0f46bdf68f5efd997d99bf8330980bb6ca7577e7f39dd" exitCode=0
Jul 24 17:35:30.505748 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:30.505716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht" event={"ID":"ff8c3642-67d5-41ce-ac9c-395053f45ab6","Type":"ContainerDied","Data":"8d38c1c9ca609bcbcca0f46bdf68f5efd997d99bf8330980bb6ca7577e7f39dd"}
Jul 24 17:35:30.506247 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:30.505793    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht" event={"ID":"ff8c3642-67d5-41ce-ac9c-395053f45ab6","Type":"ContainerStarted","Data":"6d2f9c5def860e3f5aab40d693fd9f88a735e9f59945c6ed79b0ce9672192332"}
Jul 24 17:35:31.642625 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:31.642600    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"
Jul 24 17:35:31.740212 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:31.740182    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff8c3642-67d5-41ce-ac9c-395053f45ab6-maas-api-service-ca\") pod \"ff8c3642-67d5-41ce-ac9c-395053f45ab6\" (UID: \"ff8c3642-67d5-41ce-ac9c-395053f45ab6\") "
Jul 24 17:35:31.740623 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:31.740588    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff8c3642-67d5-41ce-ac9c-395053f45ab6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff8c3642-67d5-41ce-ac9c-395053f45ab6" (UID: "ff8c3642-67d5-41ce-ac9c-395053f45ab6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:35:31.841223 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:31.841188    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff8c3642-67d5-41ce-ac9c-395053f45ab6-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:32.516626 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:32.516600    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"
Jul 24 17:35:32.516842 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:32.516593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht" event={"ID":"ff8c3642-67d5-41ce-ac9c-395053f45ab6","Type":"ContainerDied","Data":"6d2f9c5def860e3f5aab40d693fd9f88a735e9f59945c6ed79b0ce9672192332"}
Jul 24 17:35:32.516842 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:32.516725    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d2f9c5def860e3f5aab40d693fd9f88a735e9f59945c6ed79b0ce9672192332"
Jul 24 17:35:45.209791 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.209711    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"]
Jul 24 17:35:45.210317 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.209964    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st" podUID="e1035f27-51a3-4225-809a-625174c526de" containerName="maas-api" containerID="cri-o://b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a" gracePeriod=30
Jul 24 17:35:45.454556 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.454532    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:35:45.568835 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.568804    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls\") pod \"e1035f27-51a3-4225-809a-625174c526de\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") "
Jul 24 17:35:45.568992 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.568941    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1035f27-51a3-4225-809a-625174c526de-openshift-service-ca\") pod \"e1035f27-51a3-4225-809a-625174c526de\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") "
Jul 24 17:35:45.568992 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.568974    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmlvz\" (UniqueName: \"kubernetes.io/projected/e1035f27-51a3-4225-809a-625174c526de-kube-api-access-dmlvz\") pod \"e1035f27-51a3-4225-809a-625174c526de\" (UID: \"e1035f27-51a3-4225-809a-625174c526de\") "
Jul 24 17:35:45.569374 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.569321    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1035f27-51a3-4225-809a-625174c526de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1035f27-51a3-4225-809a-625174c526de" (UID: "e1035f27-51a3-4225-809a-625174c526de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:35:45.570897 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.570874    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1035f27-51a3-4225-809a-625174c526de" (UID: "e1035f27-51a3-4225-809a-625174c526de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:35:45.570998 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.570930    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1035f27-51a3-4225-809a-625174c526de-kube-api-access-dmlvz" (OuterVolumeSpecName: "kube-api-access-dmlvz") pod "e1035f27-51a3-4225-809a-625174c526de" (UID: "e1035f27-51a3-4225-809a-625174c526de"). InnerVolumeSpecName "kube-api-access-dmlvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:35:45.584742 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.584715    2569 generic.go:356] "Generic (PLEG): container finished" podID="e1035f27-51a3-4225-809a-625174c526de" containerID="b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a" exitCode=0
Jul 24 17:35:45.584867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.584763    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"
Jul 24 17:35:45.584867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.584803    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st" event={"ID":"e1035f27-51a3-4225-809a-625174c526de","Type":"ContainerDied","Data":"b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a"}
Jul 24 17:35:45.584867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.584863    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st" event={"ID":"e1035f27-51a3-4225-809a-625174c526de","Type":"ContainerDied","Data":"dcc42942bd1ffa5c58287cfcfabaa779e92098f2a2d62a3ad4ec02e0ec5129e8"}
Jul 24 17:35:45.584968 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.584890    2569 scope.go:117] "RemoveContainer" containerID="b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a"
Jul 24 17:35:45.593584 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.593525    2569 scope.go:117] "RemoveContainer" containerID="b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a"
Jul 24 17:35:45.593918 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:35:45.593848    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a\": container with ID starting with b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a not found: ID does not exist" containerID="b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a"
Jul 24 17:35:45.593918 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.593886    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a"} err="failed to get container status \"b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a\": rpc error: code = NotFound desc = could not find container \"b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a\": container with ID starting with b8ff746d711cd1ae343e19d061156d5625cf602763cf3d7fef9fe7e0cdb3964a not found: ID does not exist"
Jul 24 17:35:45.606390 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.606356    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"]
Jul 24 17:35:45.613919 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.613895    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6a1c15-7bb48c8599-bh4st"]
Jul 24 17:35:45.631040 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.631006    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1035f27-51a3-4225-809a-625174c526de" path="/var/lib/kubelet/pods/e1035f27-51a3-4225-809a-625174c526de/volumes"
Jul 24 17:35:45.669657 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.669628    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1035f27-51a3-4225-809a-625174c526de-openshift-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:45.669657 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.669653    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmlvz\" (UniqueName: \"kubernetes.io/projected/e1035f27-51a3-4225-809a-625174c526de-kube-api-access-dmlvz\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:45.669826 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:45.669665    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1035f27-51a3-4225-809a-625174c526de-maas-api-tls\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:35:49.262214 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:49.262179    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"]
Jul 24 17:35:49.267312 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:49.267283    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-6a1c15-zjtht"]
Jul 24 17:35:49.630859 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:49.630825    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff8c3642-67d5-41ce-ac9c-395053f45ab6" path="/var/lib/kubelet/pods/ff8c3642-67d5-41ce-ac9c-395053f45ab6/volumes"
Jul 24 17:35:54.634773 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:35:54.634739    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"]
Jul 24 17:36:09.653797 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.653758    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4" podUID="9efc6b5f-6900-4119-8b4f-284c445be4bf" containerName="main" containerID="cri-o://e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b" gracePeriod=45
Jul 24 17:36:09.910308 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.910247    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:36:09.993436 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993408    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-tmp-dir\") pod \"9efc6b5f-6900-4119-8b4f-284c445be4bf\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") "
Jul 24 17:36:09.993605 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993453    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efc6b5f-6900-4119-8b4f-284c445be4bf-tls-certs\") pod \"9efc6b5f-6900-4119-8b4f-284c445be4bf\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") "
Jul 24 17:36:09.993605 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993485    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-home\") pod \"9efc6b5f-6900-4119-8b4f-284c445be4bf\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") "
Jul 24 17:36:09.993605 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993520    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqgq4\" (UniqueName: \"kubernetes.io/projected/9efc6b5f-6900-4119-8b4f-284c445be4bf-kube-api-access-mqgq4\") pod \"9efc6b5f-6900-4119-8b4f-284c445be4bf\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") "
Jul 24 17:36:09.993605 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993556    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-model-cache\") pod \"9efc6b5f-6900-4119-8b4f-284c445be4bf\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") "
Jul 24 17:36:09.993817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-dshm\") pod \"9efc6b5f-6900-4119-8b4f-284c445be4bf\" (UID: \"9efc6b5f-6900-4119-8b4f-284c445be4bf\") "
Jul 24 17:36:09.993817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993687    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9efc6b5f-6900-4119-8b4f-284c445be4bf" (UID: "9efc6b5f-6900-4119-8b4f-284c445be4bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:36:09.993817 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993784    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-home" (OuterVolumeSpecName: "home") pod "9efc6b5f-6900-4119-8b4f-284c445be4bf" (UID: "9efc6b5f-6900-4119-8b4f-284c445be4bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:36:09.993977 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.993897    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-model-cache" (OuterVolumeSpecName: "model-cache") pod "9efc6b5f-6900-4119-8b4f-284c445be4bf" (UID: "9efc6b5f-6900-4119-8b4f-284c445be4bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:36:09.994035 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.994004    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-tmp-dir\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:09.994035 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.994024    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-home\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:09.994113 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.994037    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-model-cache\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:09.995683 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.995656    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9efc6b5f-6900-4119-8b4f-284c445be4bf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9efc6b5f-6900-4119-8b4f-284c445be4bf" (UID: "9efc6b5f-6900-4119-8b4f-284c445be4bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:36:09.995683 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.995677    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-dshm" (OuterVolumeSpecName: "dshm") pod "9efc6b5f-6900-4119-8b4f-284c445be4bf" (UID: "9efc6b5f-6900-4119-8b4f-284c445be4bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:36:09.995856 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:09.995687    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9efc6b5f-6900-4119-8b4f-284c445be4bf-kube-api-access-mqgq4" (OuterVolumeSpecName: "kube-api-access-mqgq4") pod "9efc6b5f-6900-4119-8b4f-284c445be4bf" (UID: "9efc6b5f-6900-4119-8b4f-284c445be4bf"). InnerVolumeSpecName "kube-api-access-mqgq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:36:10.095305 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.095265    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9efc6b5f-6900-4119-8b4f-284c445be4bf-dshm\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:10.095305 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.095294    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9efc6b5f-6900-4119-8b4f-284c445be4bf-tls-certs\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:10.095305 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.095306    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqgq4\" (UniqueName: \"kubernetes.io/projected/9efc6b5f-6900-4119-8b4f-284c445be4bf-kube-api-access-mqgq4\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:10.708419 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.708387    2569 generic.go:356] "Generic (PLEG): container finished" podID="9efc6b5f-6900-4119-8b4f-284c445be4bf" containerID="e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b" exitCode=0
Jul 24 17:36:10.708835 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.708428    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"
Jul 24 17:36:10.708835 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.708467    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4" event={"ID":"9efc6b5f-6900-4119-8b4f-284c445be4bf","Type":"ContainerDied","Data":"e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b"}
Jul 24 17:36:10.708835 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.708513    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4" event={"ID":"9efc6b5f-6900-4119-8b4f-284c445be4bf","Type":"ContainerDied","Data":"802ccfec9ade2e691fa9ac770e5319badf11611aae3399d2161da39f0ebc1d87"}
Jul 24 17:36:10.708835 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.708535    2569 scope.go:117] "RemoveContainer" containerID="e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b"
Jul 24 17:36:10.717840 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.717819    2569 scope.go:117] "RemoveContainer" containerID="e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b"
Jul 24 17:36:10.718098 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:36:10.718076    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b\": container with ID starting with e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b not found: ID does not exist" containerID="e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b"
Jul 24 17:36:10.718168 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.718107    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b"} err="failed to get container status \"e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b\": rpc error: code = NotFound desc = could not find container \"e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b\": container with ID starting with e38fd4d86012dd4ea758fb8523c8394d995591009fcc4e9dd33ec5ca2ba4200b not found: ID does not exist"
Jul 24 17:36:10.731509 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.731479    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"]
Jul 24 17:36:10.738111 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:10.738080    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-6a1c15/ipp-route-6a1c15-kserve-79bbc78959-rh5v4"]
Jul 24 17:36:11.630244 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:11.630206    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9efc6b5f-6900-4119-8b4f-284c445be4bf" path="/var/lib/kubelet/pods/9efc6b5f-6900-4119-8b4f-284c445be4bf/volumes"
Jul 24 17:36:38.230234 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230191    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"]
Jul 24 17:36:38.230867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230734    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1035f27-51a3-4225-809a-625174c526de" containerName="maas-api"
Jul 24 17:36:38.230867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230754    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1035f27-51a3-4225-809a-625174c526de" containerName="maas-api"
Jul 24 17:36:38.230867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230771    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff8c3642-67d5-41ce-ac9c-395053f45ab6" containerName="revoke-keys"
Jul 24 17:36:38.230867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230779    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff8c3642-67d5-41ce-ac9c-395053f45ab6" containerName="revoke-keys"
Jul 24 17:36:38.230867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230808    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9efc6b5f-6900-4119-8b4f-284c445be4bf" containerName="main"
Jul 24 17:36:38.230867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230818    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9efc6b5f-6900-4119-8b4f-284c445be4bf" containerName="main"
Jul 24 17:36:38.231178 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230901    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff8c3642-67d5-41ce-ac9c-395053f45ab6" containerName="revoke-keys"
Jul 24 17:36:38.231178 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230916    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9efc6b5f-6900-4119-8b4f-284c445be4bf" containerName="main"
Jul 24 17:36:38.231178 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.230927    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1035f27-51a3-4225-809a-625174c526de" containerName="maas-api"
Jul 24 17:36:38.234263 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.234243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"
Jul 24 17:36:38.239085 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.239059    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:36:38.246811 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.246787    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"]
Jul 24 17:36:38.255043 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.255023    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7bb07d-0dff-4e00-b563-a4988c053285-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg\" (UID: \"4f7bb07d-0dff-4e00-b563-a4988c053285\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"
Jul 24 17:36:38.355804 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.355774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7bb07d-0dff-4e00-b563-a4988c053285-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg\" (UID: \"4f7bb07d-0dff-4e00-b563-a4988c053285\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"
Jul 24 17:36:38.356379 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.356358    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7bb07d-0dff-4e00-b563-a4988c053285-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg\" (UID: \"4f7bb07d-0dff-4e00-b563-a4988c053285\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"
Jul 24 17:36:38.544605 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.544495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"
Jul 24 17:36:38.690911 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.690876    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"]
Jul 24 17:36:38.693303 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:36:38.693272    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f7bb07d_0dff_4e00_b563_a4988c053285.slice/crio-119b30e4f3abbe4ec6cd1a6681811d77e4d832fd8f2d2483df12c3dd0262665f WatchSource:0}: Error finding container 119b30e4f3abbe4ec6cd1a6681811d77e4d832fd8f2d2483df12c3dd0262665f: Status 404 returned error can't find the container with id 119b30e4f3abbe4ec6cd1a6681811d77e4d832fd8f2d2483df12c3dd0262665f
Jul 24 17:36:38.843975 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:38.843941    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg" event={"ID":"4f7bb07d-0dff-4e00-b563-a4988c053285","Type":"ContainerStarted","Data":"119b30e4f3abbe4ec6cd1a6681811d77e4d832fd8f2d2483df12c3dd0262665f"}
Jul 24 17:36:39.850318 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:39.850286    2569 generic.go:356] "Generic (PLEG): container finished" podID="4f7bb07d-0dff-4e00-b563-a4988c053285" containerID="c438b3dafd6112673319391abd2906be6fe852e848e0552ca32fae0f9cee38d1" exitCode=0
Jul 24 17:36:39.850782 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:39.850385    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg" event={"ID":"4f7bb07d-0dff-4e00-b563-a4988c053285","Type":"ContainerDied","Data":"c438b3dafd6112673319391abd2906be6fe852e848e0552ca32fae0f9cee38d1"}
Jul 24 17:36:40.992679 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:40.992648    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"
Jul 24 17:36:41.078034 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:41.078003    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7bb07d-0dff-4e00-b563-a4988c053285-maas-api-service-ca\") pod \"4f7bb07d-0dff-4e00-b563-a4988c053285\" (UID: \"4f7bb07d-0dff-4e00-b563-a4988c053285\") "
Jul 24 17:36:41.078424 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:41.078399    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f7bb07d-0dff-4e00-b563-a4988c053285-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f7bb07d-0dff-4e00-b563-a4988c053285" (UID: "4f7bb07d-0dff-4e00-b563-a4988c053285"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:36:41.179139 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:41.179065    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f7bb07d-0dff-4e00-b563-a4988c053285-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:41.860692 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:41.860657    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg" event={"ID":"4f7bb07d-0dff-4e00-b563-a4988c053285","Type":"ContainerDied","Data":"119b30e4f3abbe4ec6cd1a6681811d77e4d832fd8f2d2483df12c3dd0262665f"}
Jul 24 17:36:41.860692 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:41.860683    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"
Jul 24 17:36:41.860942 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:41.860705    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="119b30e4f3abbe4ec6cd1a6681811d77e4d832fd8f2d2483df12c3dd0262665f"
Jul 24 17:36:48.366491 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.366449    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"]
Jul 24 17:36:48.366996 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.366701    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr" podUID="b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" containerName="maas-api" containerID="cri-o://4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1" gracePeriod=30
Jul 24 17:36:48.610701 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.610672    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:36:48.639427 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.639344    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnzfz\" (UniqueName: \"kubernetes.io/projected/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-kube-api-access-pnzfz\") pod \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") "
Jul 24 17:36:48.639427 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.639402    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-maas-api-tls\") pod \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") "
Jul 24 17:36:48.639660 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.639495    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-openshift-service-ca\") pod \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\" (UID: \"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1\") "
Jul 24 17:36:48.639824 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.639796    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" (UID: "b30f5b34-adee-467a-b6d0-ff2c8c0a72c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:36:48.641489 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.641459    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-kube-api-access-pnzfz" (OuterVolumeSpecName: "kube-api-access-pnzfz") pod "b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" (UID: "b30f5b34-adee-467a-b6d0-ff2c8c0a72c1"). InnerVolumeSpecName "kube-api-access-pnzfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:36:48.641585 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.641521    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" (UID: "b30f5b34-adee-467a-b6d0-ff2c8c0a72c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:36:48.740548 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.740513    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnzfz\" (UniqueName: \"kubernetes.io/projected/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-kube-api-access-pnzfz\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:48.740548 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.740542    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-maas-api-tls\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:48.740548 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.740551    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1-openshift-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:36:48.896092 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.895998    2569 generic.go:356] "Generic (PLEG): container finished" podID="b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" containerID="4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1" exitCode=0
Jul 24 17:36:48.896092 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.896044    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"
Jul 24 17:36:48.896092 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.896081    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr" event={"ID":"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1","Type":"ContainerDied","Data":"4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1"}
Jul 24 17:36:48.896390 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.896133    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr" event={"ID":"b30f5b34-adee-467a-b6d0-ff2c8c0a72c1","Type":"ContainerDied","Data":"5d286c370e337b8d388fda2b92930c0dae1ed836b8e0671120248157afe56c88"}
Jul 24 17:36:48.896390 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.896157    2569 scope.go:117] "RemoveContainer" containerID="4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1"
Jul 24 17:36:48.905279 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.905264    2569 scope.go:117] "RemoveContainer" containerID="4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1"
Jul 24 17:36:48.905552 ip-10-0-135-204 kubenswrapper[2569]: E0724 17:36:48.905534    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1\": container with ID starting with 4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1 not found: ID does not exist" containerID="4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1"
Jul 24 17:36:48.905637 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.905558    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1"} err="failed to get container status \"4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1\": rpc error: code = NotFound desc = could not find container \"4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1\": container with ID starting with 4da3feed0577fa1d4bbf15725ffa55b3ded67522cfca87d87311acc3aa9ff2b1 not found: ID does not exist"
Jul 24 17:36:48.916431 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.916409    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"]
Jul 24 17:36:48.922377 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:48.922355    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a2f844-d788fc458-kwpqr"]
Jul 24 17:36:49.630589 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:49.630559    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" path="/var/lib/kubelet/pods/b30f5b34-adee-467a-b6d0-ff2c8c0a72c1/volumes"
Jul 24 17:36:53.321717 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:53.321686    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"]
Jul 24 17:36:53.325138 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:53.325108    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a2f844-4z2tg"]
Jul 24 17:36:53.629786 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:36:53.629704    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f7bb07d-0dff-4e00-b563-a4988c053285" path="/var/lib/kubelet/pods/4f7bb07d-0dff-4e00-b563-a4988c053285/volumes"
Jul 24 17:37:00.750849 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.750813    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"]
Jul 24 17:37:00.751365 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.751322    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" containerName="maas-api"
Jul 24 17:37:00.751420 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.751369    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" containerName="maas-api"
Jul 24 17:37:00.751420 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.751405    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f7bb07d-0dff-4e00-b563-a4988c053285" containerName="revoke-keys"
Jul 24 17:37:00.751420 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.751414    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f7bb07d-0dff-4e00-b563-a4988c053285" containerName="revoke-keys"
Jul 24 17:37:00.751514 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.751489    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f7bb07d-0dff-4e00-b563-a4988c053285" containerName="revoke-keys"
Jul 24 17:37:00.751514 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.751505    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30f5b34-adee-467a-b6d0-ff2c8c0a72c1" containerName="maas-api"
Jul 24 17:37:00.754464 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.754439    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"
Jul 24 17:37:00.756858 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.756836    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:37:00.756985 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.756838    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:37:00.762251 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.762166    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"]
Jul 24 17:37:00.845469 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.845433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/914b5c28-e315-44ab-a9e0-c9bf4b675e96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw\" (UID: \"914b5c28-e315-44ab-a9e0-c9bf4b675e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"
Jul 24 17:37:00.945951 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.945915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/914b5c28-e315-44ab-a9e0-c9bf4b675e96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw\" (UID: \"914b5c28-e315-44ab-a9e0-c9bf4b675e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"
Jul 24 17:37:00.946600 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:00.946583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/914b5c28-e315-44ab-a9e0-c9bf4b675e96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw\" (UID: \"914b5c28-e315-44ab-a9e0-c9bf4b675e96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"
Jul 24 17:37:01.065908 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:01.065806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"
Jul 24 17:37:01.201932 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:01.201910    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"]
Jul 24 17:37:01.204081 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:37:01.204055    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914b5c28_e315_44ab_a9e0_c9bf4b675e96.slice/crio-be6befcb79e69dcddc7147b7f21edb1021d669c98de4325f10b8860724cf9f3f WatchSource:0}: Error finding container be6befcb79e69dcddc7147b7f21edb1021d669c98de4325f10b8860724cf9f3f: Status 404 returned error can't find the container with id be6befcb79e69dcddc7147b7f21edb1021d669c98de4325f10b8860724cf9f3f
Jul 24 17:37:01.964147 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:01.964111    2569 generic.go:356] "Generic (PLEG): container finished" podID="914b5c28-e315-44ab-a9e0-c9bf4b675e96" containerID="22018bbc99dc20d964881e51088823faaa3efb2f510bb403d34b2f593b927a17" exitCode=0
Jul 24 17:37:01.964553 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:01.964186    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw" event={"ID":"914b5c28-e315-44ab-a9e0-c9bf4b675e96","Type":"ContainerDied","Data":"22018bbc99dc20d964881e51088823faaa3efb2f510bb403d34b2f593b927a17"}
Jul 24 17:37:01.964553 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:01.964241    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw" event={"ID":"914b5c28-e315-44ab-a9e0-c9bf4b675e96","Type":"ContainerStarted","Data":"be6befcb79e69dcddc7147b7f21edb1021d669c98de4325f10b8860724cf9f3f"}
Jul 24 17:37:03.107867 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:03.107840    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"
Jul 24 17:37:03.166569 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:03.166534    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/914b5c28-e315-44ab-a9e0-c9bf4b675e96-maas-api-service-ca\") pod \"914b5c28-e315-44ab-a9e0-c9bf4b675e96\" (UID: \"914b5c28-e315-44ab-a9e0-c9bf4b675e96\") "
Jul 24 17:37:03.166924 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:03.166902    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914b5c28-e315-44ab-a9e0-c9bf4b675e96-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "914b5c28-e315-44ab-a9e0-c9bf4b675e96" (UID: "914b5c28-e315-44ab-a9e0-c9bf4b675e96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:37:03.268011 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:03.267923    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/914b5c28-e315-44ab-a9e0-c9bf4b675e96-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:37:03.975895 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:03.975801    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"
Jul 24 17:37:03.975895 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:03.975839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw" event={"ID":"914b5c28-e315-44ab-a9e0-c9bf4b675e96","Type":"ContainerDied","Data":"be6befcb79e69dcddc7147b7f21edb1021d669c98de4325f10b8860724cf9f3f"}
Jul 24 17:37:03.975895 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:03.975887    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be6befcb79e69dcddc7147b7f21edb1021d669c98de4325f10b8860724cf9f3f"
Jul 24 17:37:11.366993 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:11.366916    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"]
Jul 24 17:37:11.370060 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:11.370037    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-de68c7-29748570-6kd45"]
Jul 24 17:37:11.630921 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:11.630829    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91da7613-6582-4fae-8c4b-116d58ba70f4" path="/var/lib/kubelet/pods/91da7613-6582-4fae-8c4b-116d58ba70f4/volumes"
Jul 24 17:37:14.079151 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:14.079120    2569 scope.go:117] "RemoveContainer" containerID="70f7e289c1a01922648adba4fa7c58f5a7668dec365acefc83df6793fa3f9507"
Jul 24 17:37:15.825856 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:15.825817    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"]
Jul 24 17:37:15.828567 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:15.828537    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-de68c7-pt4sw"]
Jul 24 17:37:17.629796 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:17.629766    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="914b5c28-e315-44ab-a9e0-c9bf4b675e96" path="/var/lib/kubelet/pods/914b5c28-e315-44ab-a9e0-c9bf4b675e96/volumes"
Jul 24 17:37:22.848363 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.848316    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"]
Jul 24 17:37:22.848749 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.848671    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914b5c28-e315-44ab-a9e0-c9bf4b675e96" containerName="revoke-keys"
Jul 24 17:37:22.848749 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.848682    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="914b5c28-e315-44ab-a9e0-c9bf4b675e96" containerName="revoke-keys"
Jul 24 17:37:22.848829 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.848756    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="914b5c28-e315-44ab-a9e0-c9bf4b675e96" containerName="revoke-keys"
Jul 24 17:37:22.853032 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.853014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"
Jul 24 17:37:22.862200 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.862180    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jdhj4\""
Jul 24 17:37:22.862657 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.862641    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:37:22.876254 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.876233    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"]
Jul 24 17:37:22.942651 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:22.942615    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb\" (UID: \"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"
Jul 24 17:37:23.043978 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:23.043941    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb\" (UID: \"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"
Jul 24 17:37:23.044627 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:23.044608    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb\" (UID: \"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"
Jul 24 17:37:23.162388 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:23.162264    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"
Jul 24 17:37:23.292840 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:23.292809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"]
Jul 24 17:37:23.296088 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:37:23.296051    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c890634_d2ec_4fbd_8979_7fb9d4f4b3d3.slice/crio-3e044608bb8f4634d0fc342d38a6e590936cad9398f2764dbe3805f6465e9ec1 WatchSource:0}: Error finding container 3e044608bb8f4634d0fc342d38a6e590936cad9398f2764dbe3805f6465e9ec1: Status 404 returned error can't find the container with id 3e044608bb8f4634d0fc342d38a6e590936cad9398f2764dbe3805f6465e9ec1
Jul 24 17:37:24.075589 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:24.075555    2569 generic.go:356] "Generic (PLEG): container finished" podID="9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3" containerID="1f876ded7191ec9bd88820fba6ed00e03e0adc671e76526dc5a41cad1939398e" exitCode=0
Jul 24 17:37:24.075946 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:24.075644    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb" event={"ID":"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3","Type":"ContainerDied","Data":"1f876ded7191ec9bd88820fba6ed00e03e0adc671e76526dc5a41cad1939398e"}
Jul 24 17:37:24.075946 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:24.075697    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb" event={"ID":"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3","Type":"ContainerStarted","Data":"3e044608bb8f4634d0fc342d38a6e590936cad9398f2764dbe3805f6465e9ec1"}
Jul 24 17:37:25.211052 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:25.211030    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"
Jul 24 17:37:25.365847 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:25.365743    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3-maas-api-service-ca\") pod \"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3\" (UID: \"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3\") "
Jul 24 17:37:25.366108 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:25.366083    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3" (UID: "9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:37:25.467210 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:25.467175    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3-maas-api-service-ca\") on node \"ip-10-0-135-204.ec2.internal\" DevicePath \"\""
Jul 24 17:37:26.086773 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:26.086742    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"
Jul 24 17:37:26.086947 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:26.086780    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb" event={"ID":"9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3","Type":"ContainerDied","Data":"3e044608bb8f4634d0fc342d38a6e590936cad9398f2764dbe3805f6465e9ec1"}
Jul 24 17:37:26.086947 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:26.086826    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e044608bb8f4634d0fc342d38a6e590936cad9398f2764dbe3805f6465e9ec1"
Jul 24 17:37:33.469451 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:33.469414    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"]
Jul 24 17:37:33.472752 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:33.472728    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d73e95-29748570-6nxw9"]
Jul 24 17:37:33.630192 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:33.630159    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="748df225-a88f-4049-ae58-3a0f4fb92e7d" path="/var/lib/kubelet/pods/748df225-a88f-4049-ae58-3a0f4fb92e7d/volumes"
Jul 24 17:37:37.934118 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:37.934086    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"]
Jul 24 17:37:37.939014 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:37.938990    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d73e95-vdkdb"]
Jul 24 17:37:39.629658 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:39.629625    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3" path="/var/lib/kubelet/pods/9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3/volumes"
Jul 24 17:37:45.462290 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:45.462258    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b988cc965-rpr8j_627c3596-6db9-47f3-b592-8cd25a2556b2/authorino/0.log"
Jul 24 17:37:51.737169 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:51.737137    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-769b58b787-gq457_2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7/copy-manifests/0.log"
Jul 24 17:37:51.744750 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:51.744720    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-769b58b787-gq457_2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7/manager/0.log"
Jul 24 17:37:52.329905 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:52.329877    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748555-kc6cl_9d2ca7e0-c24b-4d79-9073-5baa506b5b12/cleanup/0.log"
Jul 24 17:37:52.439786 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:52.439759    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748570-8fc7l_8517db9c-904c-4834-95fd-1b9c7380612c/cleanup/0.log"
Jul 24 17:37:53.897833 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:53.897789    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b988cc965-rpr8j_627c3596-6db9-47f3-b592-8cd25a2556b2/authorino/0.log"
Jul 24 17:37:54.506274 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:54.506243    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2r7cw_d99834be-a9c5-4309-86ed-f5d047e97c6f/manager/0.log"
Jul 24 17:37:55.444486 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:55.444453    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cb5df4894-ctb7g_7244aa23-35f3-4163-b83f-fa7fdbd6e97f/kube-auth-proxy/0.log"
Jul 24 17:37:56.284303 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:56.284273    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-7jmbd_f3e85495-a01b-406d-9294-ecd3a8096730/main/0.log"
Jul 24 17:37:56.619617 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:56.619585    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4hln_9bd74c75-ccdf-4746-a7e0-a459acf343d2/main/0.log"
Jul 24 17:37:56.843089 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:37:56.843056    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-6xbj2_039ce2dd-0db5-4c1b-aa62-b5032aa1f119/main/0.log"
Jul 24 17:38:04.796587 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:04.796553    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jmx8q_4d5c331c-2d1f-481b-a958-080182561712/global-pull-secret-syncer/0.log"
Jul 24 17:38:05.016173 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:05.016144    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sx8db_9b3b84e6-8b95-41de-89fe-b68b7158ebec/konnectivity-agent/0.log"
Jul 24 17:38:05.066432 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:05.066352    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-204.ec2.internal_aeb8794d5494e853f2db83757f0e6f8d/haproxy/0.log"
Jul 24 17:38:10.698627 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:10.698592    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b988cc965-rpr8j_627c3596-6db9-47f3-b592-8cd25a2556b2/authorino/0.log"
Jul 24 17:38:11.039735 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:11.039639    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2r7cw_d99834be-a9c5-4309-86ed-f5d047e97c6f/manager/0.log"
Jul 24 17:38:12.915767 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:12.915733    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-cbt4j_aa3d1740-2d73-48c8-bc56-66c995d90073/cluster-monitoring-operator/0.log"
Jul 24 17:38:13.059036 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.058947    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-z7m96_22bcd108-fd34-4335-9435-99d389361e21/monitoring-plugin/0.log"
Jul 24 17:38:13.261437 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.261406    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x5mq4_62611e08-01c1-4139-8308-b6236b703357/node-exporter/0.log"
Jul 24 17:38:13.279926 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.279900    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x5mq4_62611e08-01c1-4139-8308-b6236b703357/kube-rbac-proxy/0.log"
Jul 24 17:38:13.300344 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.300305    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x5mq4_62611e08-01c1-4139-8308-b6236b703357/init-textfile/0.log"
Jul 24 17:38:13.422366 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.422320    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a09a5023-7c67-4d22-a66d-06f64ce29c51/prometheus/0.log"
Jul 24 17:38:13.446068 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.446043    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a09a5023-7c67-4d22-a66d-06f64ce29c51/config-reloader/0.log"
Jul 24 17:38:13.469139 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.469108    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a09a5023-7c67-4d22-a66d-06f64ce29c51/thanos-sidecar/0.log"
Jul 24 17:38:13.490550 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.490515    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a09a5023-7c67-4d22-a66d-06f64ce29c51/kube-rbac-proxy-web/0.log"
Jul 24 17:38:13.521286 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.521263    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a09a5023-7c67-4d22-a66d-06f64ce29c51/kube-rbac-proxy/0.log"
Jul 24 17:38:13.545176 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.545157    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a09a5023-7c67-4d22-a66d-06f64ce29c51/kube-rbac-proxy-thanos/0.log"
Jul 24 17:38:13.572760 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.572739    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a09a5023-7c67-4d22-a66d-06f64ce29c51/init-config-reloader/0.log"
Jul 24 17:38:13.759208 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.759118    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7478c6548b-8lmx8_01e535b2-bb78-482b-b3bf-0db3de302090/thanos-query/0.log"
Jul 24 17:38:13.780318 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.780288    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7478c6548b-8lmx8_01e535b2-bb78-482b-b3bf-0db3de302090/kube-rbac-proxy-web/0.log"
Jul 24 17:38:13.800241 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.800215    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7478c6548b-8lmx8_01e535b2-bb78-482b-b3bf-0db3de302090/kube-rbac-proxy/0.log"
Jul 24 17:38:13.819854 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.819831    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7478c6548b-8lmx8_01e535b2-bb78-482b-b3bf-0db3de302090/prom-label-proxy/0.log"
Jul 24 17:38:13.839620 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.839596    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7478c6548b-8lmx8_01e535b2-bb78-482b-b3bf-0db3de302090/kube-rbac-proxy-rules/0.log"
Jul 24 17:38:13.860264 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:13.860231    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7478c6548b-8lmx8_01e535b2-bb78-482b-b3bf-0db3de302090/kube-rbac-proxy-metrics/0.log"
Jul 24 17:38:14.102635 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:14.102604    2569 scope.go:117] "RemoveContainer" containerID="b74647bda8f7106921eaf9428a9d03a8347164508360ad49dfe10e506e50137f"
Jul 24 17:38:15.262183 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.262150    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"]
Jul 24 17:38:15.262639 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.262527    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3" containerName="revoke-keys"
Jul 24 17:38:15.262639 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.262540    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3" containerName="revoke-keys"
Jul 24 17:38:15.262639 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.262605    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c890634-d2ec-4fbd-8979-7fb9d4f4b3d3" containerName="revoke-keys"
Jul 24 17:38:15.267321 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.267301    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.269570 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.269546    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ffwf8\"/\"openshift-service-ca.crt\""
Jul 24 17:38:15.269570 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.269561    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ffwf8\"/\"default-dockercfg-5cl2f\""
Jul 24 17:38:15.270505 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.270483    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ffwf8\"/\"kube-root-ca.crt\""
Jul 24 17:38:15.276147 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.276126    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"]
Jul 24 17:38:15.398172 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.398133    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-proc\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.398385 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.398192    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-lib-modules\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.398385 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.398273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b5ww\" (UniqueName: \"kubernetes.io/projected/0b333d4f-25ca-4e11-bff7-df2433147f22-kube-api-access-8b5ww\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.398385 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.398313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-podres\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.398522 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.398401    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-sys\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499698 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499662    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-podres\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-sys\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-proc\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499777    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-lib-modules\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b5ww\" (UniqueName: \"kubernetes.io/projected/0b333d4f-25ca-4e11-bff7-df2433147f22-kube-api-access-8b5ww\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499863    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-sys\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499864    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-podres\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.499925 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-proc\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.500166 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.499926    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b333d4f-25ca-4e11-bff7-df2433147f22-lib-modules\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.508289 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.508265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b5ww\" (UniqueName: \"kubernetes.io/projected/0b333d4f-25ca-4e11-bff7-df2433147f22-kube-api-access-8b5ww\") pod \"perf-node-gather-daemonset-6n6zk\" (UID: \"0b333d4f-25ca-4e11-bff7-df2433147f22\") " pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.578377 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.578320    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:15.711556 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:15.711528    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"]
Jul 24 17:38:15.713990 ip-10-0-135-204 kubenswrapper[2569]: W0724 17:38:15.713962    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0b333d4f_25ca_4e11_bff7_df2433147f22.slice/crio-3dba1ca6fa1c3ede1d2c56b13392ffcbd06b64ad1ad14c111837ad71eae2e610 WatchSource:0}: Error finding container 3dba1ca6fa1c3ede1d2c56b13392ffcbd06b64ad1ad14c111837ad71eae2e610: Status 404 returned error can't find the container with id 3dba1ca6fa1c3ede1d2c56b13392ffcbd06b64ad1ad14c111837ad71eae2e610
Jul 24 17:38:16.340119 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:16.340089    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk" event={"ID":"0b333d4f-25ca-4e11-bff7-df2433147f22","Type":"ContainerStarted","Data":"a02acbf50a9e30f3c583dae6b9c62c175a367bbee0e2ed11f14495bdefc4b2ac"}
Jul 24 17:38:16.340528 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:16.340137    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk" event={"ID":"0b333d4f-25ca-4e11-bff7-df2433147f22","Type":"ContainerStarted","Data":"3dba1ca6fa1c3ede1d2c56b13392ffcbd06b64ad1ad14c111837ad71eae2e610"}
Jul 24 17:38:16.340528 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:16.340165    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:16.356105 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:16.356060    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk" podStartSLOduration=1.356045715 podStartE2EDuration="1.356045715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:38:16.355173341 +0000 UTC m=+3303.114982342" watchObservedRunningTime="2026-07-24 17:38:16.356045715 +0000 UTC m=+3303.115854714"
Jul 24 17:38:17.812703 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:17.812676    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-57k6f_88ca08d7-89d0-4507-86c3-8231fa6e116a/dns/0.log"
Jul 24 17:38:17.831950 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:17.831924    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-57k6f_88ca08d7-89d0-4507-86c3-8231fa6e116a/kube-rbac-proxy/0.log"
Jul 24 17:38:17.940281 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:17.940250    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8774l_c1de7e8c-5014-4551-8158-d1c2101f616a/dns-node-resolver/0.log"
Jul 24 17:38:18.536807 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:18.536772    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-676787f6b8-s598c_061a4548-0c73-4be6-8148-69da8f83c662/registry/0.log"
Jul 24 17:38:18.570819 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:18.570793    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2mvp5_e6be9cbe-bb1c-4ba9-9aee-173c5868a4ec/node-ca/0.log"
Jul 24 17:38:19.770661 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:19.770626    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cb5df4894-ctb7g_7244aa23-35f3-4163-b83f-fa7fdbd6e97f/kube-auth-proxy/0.log"
Jul 24 17:38:20.841618 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:20.841588    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8ppx4_900c12f2-0612-4eb9-a0b4-5f93b9c8e097/serve-healthcheck-canary/0.log"
Jul 24 17:38:21.480376 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:21.480320    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g72bh_3b57bd0b-8952-4fcd-821c-db4ccfb03bbc/kube-rbac-proxy/0.log"
Jul 24 17:38:21.506199 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:21.506173    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g72bh_3b57bd0b-8952-4fcd-821c-db4ccfb03bbc/exporter/0.log"
Jul 24 17:38:21.527168 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:21.527120    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g72bh_3b57bd0b-8952-4fcd-821c-db4ccfb03bbc/extractor/0.log"
Jul 24 17:38:22.354357 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:22.354317    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ffwf8/perf-node-gather-daemonset-6n6zk"
Jul 24 17:38:24.799685 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:24.799652    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-769b58b787-gq457_2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7/manager/0.log"
Jul 24 17:38:24.836597 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:24.836571    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-769b58b787-gq457_2c77c503-ed9d-4f0d-9aaf-51ae063fd9d7/copy-manifests/0.log"
Jul 24 17:38:25.638687 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:25.638653    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cb5df4894-ctb7g_7244aa23-35f3-4163-b83f-fa7fdbd6e97f/kube-auth-proxy/0.log"
Jul 24 17:38:26.164118 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:26.164088    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-675dd78886-stwrf_90d89369-25e6-4def-a8e6-a2b24d7764f5/manager/0.log"
Jul 24 17:38:27.387534 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:27.387446    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cb5df4894-ctb7g_7244aa23-35f3-4163-b83f-fa7fdbd6e97f/kube-auth-proxy/0.log"
Jul 24 17:38:31.125195 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:31.125164    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-8tms4_9e26248c-ee8d-4c05-ae8d-98a85820e121/migrator/0.log"
Jul 24 17:38:31.145387 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:31.145363    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-8tms4_9e26248c-ee8d-4c05-ae8d-98a85820e121/graceful-termination/0.log"
Jul 24 17:38:32.505076 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:32.505048    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hzgt_44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43/kube-multus-additional-cni-plugins/0.log"
Jul 24 17:38:32.527530 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:32.527507    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hzgt_44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43/egress-router-binary-copy/0.log"
Jul 24 17:38:32.548955 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:32.548929    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hzgt_44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43/cni-plugins/0.log"
Jul 24 17:38:32.575904 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:32.575872    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hzgt_44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43/bond-cni-plugin/0.log"
Jul 24 17:38:32.597097 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:32.597067    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hzgt_44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43/routeoverride-cni/0.log"
Jul 24 17:38:32.616134 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:32.616107    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hzgt_44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43/whereabouts-cni-bincopy/0.log"
Jul 24 17:38:32.636963 ip-10-0-135-204 kubenswrapper[2569]: I0724 17:38:32.636936    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hzgt_44c14d78-a18c-4f2f-8f15-dc8bcbfbcd43/whereabouts-cni/0.log"