Jul 14 11:16:16.192175 ip-10-0-142-172 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 14 11:16:16.192188 ip-10-0-142-172 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 14 11:16:16.192194 ip-10-0-142-172 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 14 11:16:16.192435 ip-10-0-142-172 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 14 11:16:26.441407 ip-10-0-142-172 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 14 11:16:26.441427 ip-10-0-142-172 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d6bb3a767f1f4602b7bba9a01e429f3e --
Jul 14 11:18:26.087000 ip-10-0-142-172 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 11:18:26.554962 ip-10-0-142-172 systemd[1]: Started Kubernetes Kubelet.
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.480010    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483751    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483763    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483767    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483770    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483773    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483777    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483781    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:18:26.576928 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483784    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483787    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483790    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483793    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483797    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483801    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483806    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483809    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483811    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483814    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483817    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483820    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483823    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483826    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483829    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483832    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483834    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483837    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483840    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483842    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:18:26.578109 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483845    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483847    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483850    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483852    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483855    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483857    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483860    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483862    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483865    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483867    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483870    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483873    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483875    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483878    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483880    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483883    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483885    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483888    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483890    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483893    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:18:26.579102 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483895    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483897    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483901    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483903    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483907    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483910    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483912    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483915    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483918    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483920    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483923    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483926    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483928    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483931    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483933    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483936    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483938    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483941    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483943    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483946    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:18:26.580172 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483949    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483951    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483954    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483956    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483960    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483963    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483966    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483969    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483972    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483974    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483976    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483979    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483982    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483984    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483988    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483990    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483994    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.483998    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484002    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:18:26.582179 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484005    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484007    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484011    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484014    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484017    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484019    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484023    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484027    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484031    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484033    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484036    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484039    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.484042    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486100    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486108    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486111    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486114    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486117    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486121    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:18:26.583229 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486123    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486126    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486129    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486132    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486135    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486137    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486141    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486145    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486148    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486151    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486154    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486156    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486159    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486162    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486164    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486167    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486170    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486172    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486176    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:18:26.584125 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486180    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486184    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486187    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486190    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486193    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486195    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486199    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486202    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486205    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486207    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486210    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486212    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486214    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486218    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486220    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486223    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486225    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486228    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486230    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486233    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:18:26.584691 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486235    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486238    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486240    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486243    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486245    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486250    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486253    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486256    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486259    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486262    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486265    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486268    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486271    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486273    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486276    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486279    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486282    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486285    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486288    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:18:26.585286 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486290    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486293    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486295    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486298    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486301    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486303    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486306    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486312    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486315    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486319    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486321    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486324    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486326    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486329    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486331    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486334    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486336    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486338    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486341    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486343    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:18:26.585867 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486346    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486349    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486351    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486354    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486357    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486360    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486363    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486365    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486368    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486371    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486373    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486376    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486379    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486381    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486384    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486755    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486760    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486763    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486766    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486769    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:18:26.586871 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486772    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486775    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486777    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486780    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486782    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486785    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486789    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486793    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486799    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486802    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486805    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486808    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486811    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486814    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486817    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486820    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486822    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486825    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486827    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486830    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:18:26.587722 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486833    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486835    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486838    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486840    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486843    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486845    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486848    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486850    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486854    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486856    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486859    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486861    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486864    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486866    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486869    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486871    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486874    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486877    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486879    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486882    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:18:26.588590 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486887    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486890    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486893    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486895    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486898    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486901    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486905    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486908    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486910    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486913    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486915    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486918    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486920    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486923    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486925    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486928    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486930    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486933    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486935    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486937    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:18:26.589480 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486940    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486942    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486945    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486948    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486950    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486962    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486965    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486967    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486970    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486973    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486976    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486979    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486983    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486985    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486988    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486990    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486993    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.486995    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487000    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487002    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:18:26.590352 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487005    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487008    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487011    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487014    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487016    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487019    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487021    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487024    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487026    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487029    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487031    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487035    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487038    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487041    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487126    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487143    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487148    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487153    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487157    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487160    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 11:18:26.591227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487165    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487169    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487172    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487175    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487180    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487183    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487186    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487189    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487192    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487195    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487199    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487202    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487206    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487209    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487212    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487215    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487219    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487222    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487225    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487229    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487232    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487235    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487238    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487241    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 11:18:26.592319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487244    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487247    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487251    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487254    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487257    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487260    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487263    2564 flags.go:64] FLAG: --enable-server="true"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487266    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487270    2564 flags.go:64] FLAG: --event-burst="100"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487273    2564 flags.go:64] FLAG: --event-qps="50"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487276    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487279    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487283    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487287    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487290    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487292    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487295    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487298    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487303    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487305    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487308    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487311    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487314    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487317    2564 flags.go:64] FLAG: --feature-gates=""
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487320    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 11:18:26.593395 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487323    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487326    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487330    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487333    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487335    2564 flags.go:64] FLAG: --help="false"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487338    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-142-172.ec2.internal"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487341    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487344    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487347    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487351    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487354    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487356    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487360    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487363    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487365    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487369    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487372    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487374    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487377    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487381    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487384    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487387    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487390    2564 flags.go:64] FLAG: --lock-file=""
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487393    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 11:18:26.594482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487396    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487400    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487406    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487408    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487411    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487414    2564 flags.go:64] FLAG: --logging-format="text"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487417    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487420    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487423    2564 flags.go:64] FLAG: --manifest-url=""
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487426    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487430    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487433    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487438    2564 flags.go:64] FLAG: --max-pods="110"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487440    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487443    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487446    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487449    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487452    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487455    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487458    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487467    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487470    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487473    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 11:18:26.597666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487476    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487480    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487485    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487488    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487491    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487495    2564 flags.go:64] FLAG: --port="10250"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487498    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487501    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ba7c3be001bba745"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487504    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487510    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487514    2564 flags.go:64] FLAG: --register-node="true"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487517    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487520    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487523    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487526    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487528    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487532    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487535    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487538    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487540    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487543    2564 flags.go:64] FLAG: --runonce="false"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487546    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487549    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487552    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487555    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 11:18:26.598965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487558    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487561    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487564    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487567    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487570    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487573    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487576    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487578    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487582    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487585    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487588    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487593    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487597    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487599    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487606    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487609    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487612    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487616    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487619    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487622    2564 flags.go:64] FLAG: --v="2"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487625    2564 flags.go:64] FLAG: --version="false"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487629    2564 flags.go:64] FLAG: --vmodule=""
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487632    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.487636    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 11:18:26.600457 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487769    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487774    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487777    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487780    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487783    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487786    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487789    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487792    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487794    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487797    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487800    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487802    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487805    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487807    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487810    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487813    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487816    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487818    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487822    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487824    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:18:26.601557 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487830    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487833    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487835    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487838    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487840    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487844    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487847    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487850    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487852    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487855    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487857    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487860    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487863    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487866    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487868    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487871    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487873    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487876    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487878    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487881    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:18:26.824787 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487883    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487885    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487888    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487891    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487893    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487895    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487898    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487901    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487903    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487906    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487908    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487911    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487915    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487917    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487920    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487922    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487925    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487929    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487931    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487934    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:18:26.845594 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487937    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487940    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487944    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487947    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487950    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487952    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487955    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487957    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487960    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487962    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487965    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487969    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487972    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487975    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487978    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487981    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487984    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487987    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487989    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:18:26.846734 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487992    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487995    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.487998    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488001    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488004    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488008    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488011    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488013    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488016    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488019    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488022    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488025    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488027    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488030    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488032    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488035    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488037    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488040    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488042    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:18:26.847376 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.488045    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.488755    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.496219    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.496233    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.496252    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.496258    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496314    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496319    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496322    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496325    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496329    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496332    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496334    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496337    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496339    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:18:26.848235 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496342    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496344    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496348    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496350    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496353    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496356    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496359    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496362    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496364    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496367    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496371    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496375    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496378    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496381    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496383    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496386    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496389    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496391    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496394    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496397    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:18:26.848749 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496400    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496403    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496405    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496408    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496410    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496413    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496415    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496417    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496420    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496423    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496425    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496428    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496430    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496433    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496436    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496438    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496442    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496444    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496447    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496450    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:18:26.849468 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496452    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496455    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496458    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496460    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496463    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496466    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496470    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496473    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496476    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496480    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496483    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496486    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496489    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496493    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496495    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496498    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496501    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496503    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496506    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:18:26.850580 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496508    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496511    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496514    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496516    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496519    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496521    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496524    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496526    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496529    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496531    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496533    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496536    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496539    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496541    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496544    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496546    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496549    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496551    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496556    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496559    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:18:26.851190 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496564    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496566    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496569    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496571    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496575    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496578    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496582    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496585    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496588    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496591    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496594    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.496599    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496728    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496734    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496737    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 11:18:26.851827 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496740    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496743    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496745    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496748    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496750    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496753    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496756    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496758    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496761    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496763    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496766    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496770    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496774    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496776    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496779    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496782    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496785    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496790    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496792    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 11:18:27.067717 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496795    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496797    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496800    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496803    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496806    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496808    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496812    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496814    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496817    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496819    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496821    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496824    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496827    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496829    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496832    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496834    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496837    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496839    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496842    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496844    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496847    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 11:18:27.069334 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496849    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496852    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496854    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496857    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496859    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496862    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496865    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496868    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496873    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496877    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496880    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496883    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496886    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496889    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496891    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496894    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496896    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496899    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496902    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 11:18:27.070494 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496905    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496907    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496910    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496912    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496915    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496917    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496920    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496923    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496925    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496927    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496930    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496933    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496935    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496938    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496940    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496943    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496946    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496948    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496952    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496955    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 11:18:27.072815 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496958    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496962    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496966    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496968    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496971    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496973    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496976    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496978    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496981    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496984    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496986    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496989    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496992    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496994    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496997    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.496999    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 11:18:27.073992 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:26.497002    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.497007    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.497104    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.499305    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.500937    2564 server.go:1019] "Starting client certificate rotation"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.501044    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.501094    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.503582    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.508212    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.519259    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.521411    2564 log.go:25] "Validated CRI v1 image API"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.522204    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 11:18:27.076074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.529136    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7f23fc01-5c52-408a-9533-73598d222a90:/dev/nvme0n1p3 974e9656-62db-4701-91d4-b4526053907a:/dev/nvme0n1p4]
Jul 14 11:18:27.076763 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.529150    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.534226    2564 manager.go:216] Machine: {Timestamp:2026-07-14 11:18:26.533768626 +0000 UTC m=+0.334324358 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100268 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27e3d6c8c801d560c509a4aea76bf1 SystemUUID:ec27e3d6-c8c8-01d5-60c5-09a4aea76bf1 BootID:d6bb3a76-7f1f-4602-b7bb-a9a01e429f3e Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9e:b4:79:ed:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9e:b4:79:ed:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:2d:3d:69:6b:f1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.534321    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.534390    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.537338    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.537364    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-172.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.537952    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.537962    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 11:18:27.077691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.537971    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.539181    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.540282    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.540599    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.543506    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.543627    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.543660    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.543680    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.543713    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.543726    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.543737    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.544124    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.544152    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.547489    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549093    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549120    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549831    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549845    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549850    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549856    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549864    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549870    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549878    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549884    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549891    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549896    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549906    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.549912    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.551054    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.551064    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.554098    2564 server.go:1268] "Started kubelet"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.554187    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.554180    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.554242    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.555717    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.557089    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.557209    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.557493    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-172.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.557722    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 11:18:27.078384 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.559820    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.559788    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-172.ec2.internal.18c223edc789ff67  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-172.ec2.internal,UID:ip-10-0-142-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-172.ec2.internal,},FirstTimestamp:2026-07-14 11:18:26.554068839 +0000 UTC m=+0.354624569,LastTimestamp:2026-07-14 11:18:26.554068839 +0000 UTC m=+0.354624569,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-172.ec2.internal,}"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.563246    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.563331    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.563463    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564023    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564026    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564050    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564129    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564151    2564 factory.go:55] Registering systemd factory
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564159    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564250    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564261    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564391    2564 factory.go:153] Registering CRI-O factory
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564404    2564 factory.go:223] Registration of the crio container factory successfully
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564426    2564 factory.go:103] Registering Raw factory
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.564441    2564 manager.go:1196] Started watching for new ooms in manager
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.565124    2564 manager.go:349] Starting recovery of all containers
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.566891    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.567081    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.570608    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lx8wx"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.570800    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.581620    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lx8wx"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.582366    2564 manager.go:354] Recovery completed
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.587271    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.588992    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.589026    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.589040    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.589597    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.589609    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.589627    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.589646    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.590332    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:18:27.080049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.590357    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.590370    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.590684    2564 policy_none.go:49] "None policy: Start"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.590716    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.590729    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.591385    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-172.ec2.internal.18c223edc99f20df  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-172.ec2.internal,UID:ip-10-0-142-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-172.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-172.ec2.internal,},FirstTimestamp:2026-07-14 11:18:26.589008095 +0000 UTC m=+0.389563833,LastTimestamp:2026-07-14 11:18:26.589008095 +0000 UTC m=+0.389563833,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-172.ec2.internal,}"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.591734    2564 policy_none.go:47] "Start"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.639158    2564 manager.go:337] "Starting Device Plugin manager"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.639183    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.639192    2564 server.go:87] "Starting device plugin registration server"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.639398    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.639408    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.639516    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.639596    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.639605    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.640042    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.640086    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.743481    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.744414    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.744446    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.744458    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.744487    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.753720    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.753744    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-172.ec2.internal\": node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.763181    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.763218    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.763238    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.763271    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.765943    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.769532    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.867979    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal"]
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.868039    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:18:27.081676 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.869566    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.869602    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.869632    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.869646    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.871308    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.871430    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.871460    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.872066    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.872116    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.872129    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.872066    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.872168    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.872188    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.873672    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.873693    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.874334    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.874366    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.874382    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeHasSufficientPID"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.905470    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-172.ec2.internal\" not found" node="ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.912760    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-172.ec2.internal\" not found" node="ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.968624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d16d29bd7cb1367aab205ff1ba54f40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal\" (UID: \"6d16d29bd7cb1367aab205ff1ba54f40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.968646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d16d29bd7cb1367aab205ff1ba54f40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal\" (UID: \"6d16d29bd7cb1367aab205ff1ba54f40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:26.968668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdebd314971aba3dd18c52401ee8209e-config\") pod \"kube-apiserver-proxy-ip-10-0-142-172.ec2.internal\" (UID: \"bdebd314971aba3dd18c52401ee8209e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:26.969737    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.069762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d16d29bd7cb1367aab205ff1ba54f40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal\" (UID: \"6d16d29bd7cb1367aab205ff1ba54f40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.083106 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:27.069778    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.084304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.069793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d16d29bd7cb1367aab205ff1ba54f40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal\" (UID: \"6d16d29bd7cb1367aab205ff1ba54f40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.084304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.069817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdebd314971aba3dd18c52401ee8209e-config\") pod \"kube-apiserver-proxy-ip-10-0-142-172.ec2.internal\" (UID: \"bdebd314971aba3dd18c52401ee8209e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.084304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.069817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6d16d29bd7cb1367aab205ff1ba54f40-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal\" (UID: \"6d16d29bd7cb1367aab205ff1ba54f40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.084304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.069852    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d16d29bd7cb1367aab205ff1ba54f40-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal\" (UID: \"6d16d29bd7cb1367aab205ff1ba54f40\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.084304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.069904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdebd314971aba3dd18c52401ee8209e-config\") pod \"kube-apiserver-proxy-ip-10-0-142-172.ec2.internal\" (UID: \"bdebd314971aba3dd18c52401ee8209e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.294564 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:27.170753    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.294564 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.210958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.294564 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.216750    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.294564 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:27.271325    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.501407 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:27.372297    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.501407 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:27.473021    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.503281 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.502385    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 11:18:27.503281 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.502532    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 11:18:27.680689 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.545996    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:18:27.680689 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:27.573073    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.680689 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.587899    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 11:13:27 +0000 UTC" deadline="2028-04-25 07:20:40.280655066 +0000 UTC"
Jul 14 11:18:27.680689 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.587923    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15620h2m12.692734816s"
Jul 14 11:18:27.680689 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:27.673778    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-172.ec2.internal\" not found"
Jul 14 11:18:27.820471 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.740732    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:18:27.820471 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.764098    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.820471 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.776530    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 11:18:27.820471 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.778164    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal"
Jul 14 11:18:27.820471 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.788423    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 11:18:27.992248 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:27.927395    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d16d29bd7cb1367aab205ff1ba54f40.slice/crio-354ed21824e41439f663cf71c7385b0fc7ea1fc07943c0131d6143af179d678e WatchSource:0}: Error finding container 354ed21824e41439f663cf71c7385b0fc7ea1fc07943c0131d6143af179d678e: Status 404 returned error can't find the container with id 354ed21824e41439f663cf71c7385b0fc7ea1fc07943c0131d6143af179d678e
Jul 14 11:18:27.992248 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:27.932638    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:18:27.992248 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:27.939496    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdebd314971aba3dd18c52401ee8209e.slice/crio-f7752df7eead7e6cbf3830ac3708e5030a65300845d0c6082795a02243143aea WatchSource:0}: Error finding container f7752df7eead7e6cbf3830ac3708e5030a65300845d0c6082795a02243143aea: Status 404 returned error can't find the container with id f7752df7eead7e6cbf3830ac3708e5030a65300845d0c6082795a02243143aea
Jul 14 11:18:28.181380 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.090967    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:18:28.545628 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.545598    2564 apiserver.go:52] "Watching apiserver"
Jul 14 11:18:28.555885 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.555433    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 11:18:28.557889 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.557420    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-kl6lg","kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal","openshift-image-registry/node-ca-bl2sn","openshift-multus/network-metrics-daemon-89z9c","openshift-network-operator/iptables-alerter-jzknk","openshift-ovn-kubernetes/ovnkube-node-d5lm4","kube-system/global-pull-secret-syncer-cbj6p","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5","openshift-cluster-node-tuning-operator/tuned-qnrnz","openshift-dns/node-resolver-dgrpv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal","openshift-multus/multus-5mg9c","openshift-multus/multus-additional-cni-plugins-l5sjl","openshift-network-diagnostics/network-check-target-zp7vf"]
Jul 14 11:18:28.562175 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.562142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.562268 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.562222    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:28.564637 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.564403    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.566912 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.566890    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 11:18:28.567070 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.567045    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.569196 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.569162    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:28.569302 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.569240    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:28.572134 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.572112    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.574055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.572982    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 11:18:28.574055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.573215    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 11:18:28.574055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.573429    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 11:18:28.574055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.573619    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 11:18:28.574055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.573686    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qv2md\""
Jul 14 11:18:28.574055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.573937    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 11:18:28.574055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.574025    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 11:18:28.574760 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.574180    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 11:18:28.574760 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.574416    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zqx8c\""
Jul 14 11:18:28.574760 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.574753    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 11:18:28.575010 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.574804    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-86q65\""
Jul 14 11:18:28.575072 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.575008    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:18:28.575305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.575285    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 11:18:28.575305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.575302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.575454 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.575439    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 11:18:28.577884 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.577864    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.579078 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-cnibin\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579181 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579101    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szkkf\" (UniqueName: \"kubernetes.io/projected/de80162a-fb30-416b-bda8-c41c436698b1-kube-api-access-szkkf\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579181 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-host\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.579297 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-os-release\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579297 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579297 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579272    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0ffcf77-e340-4c18-96d2-c069f045d964-kubelet-config\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579296    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-system-cni-dir\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0ffcf77-e340-4c18-96d2-c069f045d964-dbus\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-serviceca\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.579499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8m4c\" (UniqueName: \"kubernetes.io/projected/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-kube-api-access-s8m4c\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.579974 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.579949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.582919 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.582898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:28.583023 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.582981    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:28.584209 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.584142    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 11:18:28.584209 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.584148    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 11:18:28.584375 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.584314    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 11:18:28.585619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.585599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.585958 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.585940    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586018    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586069    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586024    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586148    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586255    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-st6tx\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586355    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586640    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586679    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2btcc\""
Jul 14 11:18:28.586817 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.586777    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dxzcl\""
Jul 14 11:18:28.587772 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.587644    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dhqpf\""
Jul 14 11:18:28.589269 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.588766    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 11:18:28.589269 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.588887    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.589269 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.588919    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 11:18:28.591476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.590931    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 11:18:28.591476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.590959    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 11:18:28.591476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.591099    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 11:18:28.591476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.591219    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 11:18:28.591476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.591253    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.591476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.591329    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gslqb\""
Jul 14 11:18:28.593110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.593012    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 11:18:28.593406 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.593233    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fst6j\""
Jul 14 11:18:28.619773 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.618744    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sjzwf"
Jul 14 11:18:28.624859 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.624473    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sjzwf"
Jul 14 11:18:28.666028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.666007    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 11:18:28.679671 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679644    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 11:18:28.679807 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-socket-dir-parent\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.679807 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ec4cdd-1108-4609-b041-5140ab4972f4-iptables-alerter-script\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.679807 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679761    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.679807 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-kubelet\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-systemd\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679835    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-cni-bin\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679860    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysctl-conf\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679886    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-systemd\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-run\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.679988    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-systemd-units\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.680028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-node-log\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.680438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680032    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-etc-selinux\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.680438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680056    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-sys-fs\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.680438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680082    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-modprobe-d\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.680438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680111    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8m4c\" (UniqueName: \"kubernetes.io/projected/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-kube-api-access-s8m4c\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.680438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-env-overrides\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.680703 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.680703 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680548    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/531c861d-fdf2-4713-942d-c18700ba8aa5-hosts-file\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.680703 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680590    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6036834-7cc4-4b56-8044-4fc362b51b24-cni-binary-copy\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.680703 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680602    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.680703 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680622    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-conf-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.680703 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:28.680703 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680697    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-cnibin\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680722    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-system-cni-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-etc-kubernetes\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680811    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-cnibin\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680857    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqktv\" (UniqueName: \"kubernetes.io/projected/ac71ccb4-b649-43d7-a85c-bd783e16364e-kube-api-access-zqktv\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-os-release\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680972    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovnkube-config\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.680998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-os-release\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681004    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-os-release\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681022    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-netns\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681056    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-cni-bin\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-cni-multus\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8wnk\" (UniqueName: \"kubernetes.io/projected/ab6dd4be-d083-4d37-b6b8-16abfcb09760-kube-api-access-p8wnk\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0ffcf77-e340-4c18-96d2-c069f045d964-kubelet-config\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.681529 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-slash\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681290    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/531c861d-fdf2-4713-942d-c18700ba8aa5-tmp-dir\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681319    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0ffcf77-e340-4c18-96d2-c069f045d964-kubelet-config\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681363    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-daemon-config\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681393    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62wn7\" (UniqueName: \"kubernetes.io/projected/a6036834-7cc4-4b56-8044-4fc362b51b24-kube-api-access-62wn7\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0ffcf77-e340-4c18-96d2-c069f045d964-dbus\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-etc-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681472    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysconfig\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681499    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-var-lib-kubelet\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szkkf\" (UniqueName: \"kubernetes.io/projected/de80162a-fb30-416b-bda8-c41c436698b1-kube-api-access-szkkf\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681545    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-run-netns\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.682244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-tuned\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-run-ovn-kubernetes\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0ffcf77-e340-4c18-96d2-c069f045d964-dbus\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-cni-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-lib-modules\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de80162a-fb30-416b-bda8-c41c436698b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-serviceca\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681900    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681952    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-registration-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.681977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-multus-certs\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysctl-d\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-log-socket\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.683043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682072    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovnkube-script-lib\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2td5g\" (UniqueName: \"kubernetes.io/projected/a523d557-2ff6-448e-a574-dd798eea5423-kube-api-access-2td5g\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682131    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plndg\" (UniqueName: \"kubernetes.io/projected/46ec4cdd-1108-4609-b041-5140ab4972f4-kube-api-access-plndg\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682164    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-kubernetes\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-host\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-var-lib-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682269    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-device-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682192    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-serviceca\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682312    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-cnibin\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-hostroot\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/19adb3db-cb83-433e-bf71-997f74e2c60f-agent-certs\") pod \"konnectivity-agent-kl6lg\" (UID: \"19adb3db-cb83-433e-bf71-997f74e2c60f\") " pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-kubelet\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682458    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-sys\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.683686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac71ccb4-b649-43d7-a85c-bd783e16364e-tmp\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682540    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-system-cni-dir\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/19adb3db-cb83-433e-bf71-997f74e2c60f-konnectivity-ca\") pod \"konnectivity-agent-kl6lg\" (UID: \"19adb3db-cb83-433e-bf71-997f74e2c60f\") " pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.682684    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs4t7\" (UniqueName: \"kubernetes.io/projected/442f870e-3885-472d-8215-b2a5eeaa9a3a-kube-api-access-bs4t7\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5lsq\" (UniqueName: \"kubernetes.io/projected/531c861d-fdf2-4713-942d-c18700ba8aa5-kube-api-access-q5lsq\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.682802    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret podName:e0ffcf77-e340-4c18-96d2-c069f045d964 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:29.182769153 +0000 UTC m=+2.983324914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret") pod "global-pull-secret-syncer-cbj6p" (UID: "e0ffcf77-e340-4c18-96d2-c069f045d964") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-k8s-cni-cncf-io\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682829    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de80162a-fb30-416b-bda8-c41c436698b1-system-cni-dir\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682859    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ec4cdd-1108-4609-b041-5140ab4972f4-host-slash\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-host\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682923    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-ovn\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682970    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-cni-netd\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.684400 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.682992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovn-node-metrics-cert\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.685137 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.683021    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-socket-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.685137 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.683054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-host\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.692298 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.692241    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 11:18:28.697640 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.697575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szkkf\" (UniqueName: \"kubernetes.io/projected/de80162a-fb30-416b-bda8-c41c436698b1-kube-api-access-szkkf\") pod \"multus-additional-cni-plugins-l5sjl\" (UID: \"de80162a-fb30-416b-bda8-c41c436698b1\") " pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.698198 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.698156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8m4c\" (UniqueName: \"kubernetes.io/projected/c3ca13a8-f8ad-4565-b6f9-f3a5342e3998-kube-api-access-s8m4c\") pod \"node-ca-bl2sn\" (UID: \"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998\") " pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.769130 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.769050    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal" event={"ID":"bdebd314971aba3dd18c52401ee8209e","Type":"ContainerStarted","Data":"f7752df7eead7e6cbf3830ac3708e5030a65300845d0c6082795a02243143aea"}
Jul 14 11:18:28.770733 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.770668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal" event={"ID":"6d16d29bd7cb1367aab205ff1ba54f40","Type":"ContainerStarted","Data":"354ed21824e41439f663cf71c7385b0fc7ea1fc07943c0131d6143af179d678e"}
Jul 14 11:18:28.783875 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.783846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8wnk\" (UniqueName: \"kubernetes.io/projected/ab6dd4be-d083-4d37-b6b8-16abfcb09760-kube-api-access-p8wnk\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:28.784030 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.783881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-slash\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.784030 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.783908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.784030 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.783935    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/531c861d-fdf2-4713-942d-c18700ba8aa5-tmp-dir\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.784030 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.783963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-daemon-config\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.784030 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.783995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62wn7\" (UniqueName: \"kubernetes.io/projected/a6036834-7cc4-4b56-8044-4fc362b51b24-kube-api-access-62wn7\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.784030 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784009    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-etc-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784055    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-etc-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysconfig\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-var-lib-kubelet\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-run-netns\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-tuned\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-run-ovn-kubernetes\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784191    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-cni-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784214    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-lib-modules\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784228    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-var-lib-kubelet\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-registration-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.784350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-multus-certs\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysctl-d\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/531c861d-fdf2-4713-942d-c18700ba8aa5-tmp-dir\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-log-socket\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovnkube-script-lib\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2td5g\" (UniqueName: \"kubernetes.io/projected/a523d557-2ff6-448e-a574-dd798eea5423-kube-api-access-2td5g\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784492    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-run-netns\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784492    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plndg\" (UniqueName: \"kubernetes.io/projected/46ec4cdd-1108-4609-b041-5140ab4972f4-kube-api-access-plndg\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-kubernetes\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-host\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784587    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-var-lib-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784652    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-device-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-cnibin\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.785120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-hostroot\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/19adb3db-cb83-433e-bf71-997f74e2c60f-agent-certs\") pod \"konnectivity-agent-kl6lg\" (UID: \"19adb3db-cb83-433e-bf71-997f74e2c60f\") " pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784746    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-kubelet\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-sys\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysconfig\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac71ccb4-b649-43d7-a85c-bd783e16364e-tmp\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/19adb3db-cb83-433e-bf71-997f74e2c60f-konnectivity-ca\") pod \"konnectivity-agent-kl6lg\" (UID: \"19adb3db-cb83-433e-bf71-997f74e2c60f\") " pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784908    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs4t7\" (UniqueName: \"kubernetes.io/projected/442f870e-3885-472d-8215-b2a5eeaa9a3a-kube-api-access-bs4t7\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5lsq\" (UniqueName: \"kubernetes.io/projected/531c861d-fdf2-4713-942d-c18700ba8aa5-kube-api-access-q5lsq\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784966    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-k8s-cni-cncf-io\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ec4cdd-1108-4609-b041-5140ab4972f4-host-slash\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-ovn\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-cni-netd\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785105    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-cni-netd\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785198    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-cnibin\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.785845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785257    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-kubernetes\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-var-lib-openvswitch\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-host\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-registration-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785404    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovn-node-metrics-cert\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-hostroot\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-socket-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-socket-dir-parent\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ec4cdd-1108-4609-b041-5140ab4972f4-iptables-alerter-script\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785569    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-kubelet\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-kubelet\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785824    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysctl-d\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785894    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-k8s-cni-cncf-io\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.786587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-log-socket\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.785981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/46ec4cdd-1108-4609-b041-5140ab4972f4-host-slash\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-ovn\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786061    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/19adb3db-cb83-433e-bf71-997f74e2c60f-konnectivity-ca\") pod \"konnectivity-agent-kl6lg\" (UID: \"19adb3db-cb83-433e-bf71-997f74e2c60f\") " pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786071    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-multus-certs\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-slash\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.784765    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-lib-modules\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-systemd\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786196    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-kubelet\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-cni-bin\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysctl-conf\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786291    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-socket-dir-parent\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-systemd\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-run\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.787305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786363    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-run-ovn-kubernetes\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-socket-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-run-systemd\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786424    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-systemd-units\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-run\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-device-dir\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786522    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-systemd\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-systemd-units\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-sysctl-conf\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-node-log\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-host-cni-bin\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-etc-selinux\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/442f870e-3885-472d-8215-b2a5eeaa9a3a-node-log\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786697    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-etc-selinux\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-sys-fs\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.788063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-modprobe-d\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-sys\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-env-overrides\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/531c861d-fdf2-4713-942d-c18700ba8aa5-hosts-file\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6036834-7cc4-4b56-8044-4fc362b51b24-cni-binary-copy\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786907    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-conf-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-system-cni-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786998    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-etc-kubernetes\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqktv\" (UniqueName: \"kubernetes.io/projected/ac71ccb4-b649-43d7-a85c-bd783e16364e-kube-api-access-zqktv\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787076    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/46ec4cdd-1108-4609-b041-5140ab4972f4-iptables-alerter-script\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovnkube-config\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787130    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-os-release\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.787187    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.787337    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs podName:ab6dd4be-d083-4d37-b6b8-16abfcb09760 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:29.287313154 +0000 UTC m=+3.087868907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs") pod "network-metrics-daemon-89z9c" (UID: "ab6dd4be-d083-4d37-b6b8-16abfcb09760") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:28.788872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787594    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/531c861d-fdf2-4713-942d-c18700ba8aa5-hosts-file\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.786738    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-cni-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-env-overrides\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787955    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-modprobe-d\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.788169    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-netns\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.788231    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-system-cni-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.788241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-cni-bin\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.788290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-cni-multus\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.788357    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovnkube-script-lib\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.788292    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-etc-kubernetes\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.789438    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac71ccb4-b649-43d7-a85c-bd783e16364e-etc-tuned\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.789571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/19adb3db-cb83-433e-bf71-997f74e2c60f-agent-certs\") pod \"konnectivity-agent-kl6lg\" (UID: \"19adb3db-cb83-433e-bf71-997f74e2c60f\") " pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.789645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.789615    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6036834-7cc4-4b56-8044-4fc362b51b24-cni-binary-copy\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.790331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac71ccb4-b649-43d7-a85c-bd783e16364e-tmp\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.790331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovnkube-config\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.790331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.787786    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a523d557-2ff6-448e-a574-dd798eea5423-sys-fs\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.790331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.788416    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-conf-dir\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.790331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/442f870e-3885-472d-8215-b2a5eeaa9a3a-ovn-node-metrics-cert\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.790331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-run-netns\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.790631 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-os-release\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.790631 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790412    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-cni-bin\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.790631 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a6036834-7cc4-4b56-8044-4fc362b51b24-host-var-lib-cni-multus\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.790631 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.790544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a6036834-7cc4-4b56-8044-4fc362b51b24-multus-daemon-config\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.796129 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.795457    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:18:28.796129 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.795486    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:18:28.796129 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.795503    2564 projected.go:196] Error preparing data for projected volume kube-api-access-vpj89 for pod openshift-network-diagnostics/network-check-target-zp7vf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:28.796129 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:28.795611    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89 podName:fed8361b-4c48-4a30-8c36-8349ef0667f2 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:29.295592471 +0000 UTC m=+3.096148225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vpj89" (UniqueName: "kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89") pod "network-check-target-zp7vf" (UID: "fed8361b-4c48-4a30-8c36-8349ef0667f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:28.799382 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.799352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plndg\" (UniqueName: \"kubernetes.io/projected/46ec4cdd-1108-4609-b041-5140ab4972f4-kube-api-access-plndg\") pod \"iptables-alerter-jzknk\" (UID: \"46ec4cdd-1108-4609-b041-5140ab4972f4\") " pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.799874 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.799851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62wn7\" (UniqueName: \"kubernetes.io/projected/a6036834-7cc4-4b56-8044-4fc362b51b24-kube-api-access-62wn7\") pod \"multus-5mg9c\" (UID: \"a6036834-7cc4-4b56-8044-4fc362b51b24\") " pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:28.799970 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.799903    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2td5g\" (UniqueName: \"kubernetes.io/projected/a523d557-2ff6-448e-a574-dd798eea5423-kube-api-access-2td5g\") pod \"aws-ebs-csi-driver-node-h6bg5\" (UID: \"a523d557-2ff6-448e-a574-dd798eea5423\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.800800 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.800730    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqktv\" (UniqueName: \"kubernetes.io/projected/ac71ccb4-b649-43d7-a85c-bd783e16364e-kube-api-access-zqktv\") pod \"tuned-qnrnz\" (UID: \"ac71ccb4-b649-43d7-a85c-bd783e16364e\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.800800 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.800740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5lsq\" (UniqueName: \"kubernetes.io/projected/531c861d-fdf2-4713-942d-c18700ba8aa5-kube-api-access-q5lsq\") pod \"node-resolver-dgrpv\" (UID: \"531c861d-fdf2-4713-942d-c18700ba8aa5\") " pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.801988 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.801960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8wnk\" (UniqueName: \"kubernetes.io/projected/ab6dd4be-d083-4d37-b6b8-16abfcb09760-kube-api-access-p8wnk\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:28.803162 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.803125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs4t7\" (UniqueName: \"kubernetes.io/projected/442f870e-3885-472d-8215-b2a5eeaa9a3a-kube-api-access-bs4t7\") pod \"ovnkube-node-d5lm4\" (UID: \"442f870e-3885-472d-8215-b2a5eeaa9a3a\") " pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.881644 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.881605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l5sjl"
Jul 14 11:18:28.891770 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.891735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bl2sn"
Jul 14 11:18:28.895348 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:28.895321    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde80162a_fb30_416b_bda8_c41c436698b1.slice/crio-6ce365689c4be0098288b831f24e9fec0e66c3707f0783376ebe9590d0770584 WatchSource:0}: Error finding container 6ce365689c4be0098288b831f24e9fec0e66c3707f0783376ebe9590d0770584: Status 404 returned error can't find the container with id 6ce365689c4be0098288b831f24e9fec0e66c3707f0783376ebe9590d0770584
Jul 14 11:18:28.903126 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.903102    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jzknk"
Jul 14 11:18:28.903744 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:28.903691    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ca13a8_f8ad_4565_b6f9_f3a5342e3998.slice/crio-2c50c166686ba8d98d1c8b47c0b1cb9d3d83ae08db78db1906768bd8ce1c1216 WatchSource:0}: Error finding container 2c50c166686ba8d98d1c8b47c0b1cb9d3d83ae08db78db1906768bd8ce1c1216: Status 404 returned error can't find the container with id 2c50c166686ba8d98d1c8b47c0b1cb9d3d83ae08db78db1906768bd8ce1c1216
Jul 14 11:18:28.913427 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.913405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:28.916390 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:28.916349    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ec4cdd_1108_4609_b041_5140ab4972f4.slice/crio-7ef229f44b40ff9ef67cbe8295eee4651b649b483534eff1443a6eef95d440d1 WatchSource:0}: Error finding container 7ef229f44b40ff9ef67cbe8295eee4651b649b483534eff1443a6eef95d440d1: Status 404 returned error can't find the container with id 7ef229f44b40ff9ef67cbe8295eee4651b649b483534eff1443a6eef95d440d1
Jul 14 11:18:28.924509 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:28.924482    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442f870e_3885_472d_8215_b2a5eeaa9a3a.slice/crio-f70a6f929ea595658ac21b5a087363fff082c83e092b3ff7a18d8f91c42181cf WatchSource:0}: Error finding container f70a6f929ea595658ac21b5a087363fff082c83e092b3ff7a18d8f91c42181cf: Status 404 returned error can't find the container with id f70a6f929ea595658ac21b5a087363fff082c83e092b3ff7a18d8f91c42181cf
Jul 14 11:18:28.926329 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.926304    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnrnz"
Jul 14 11:18:28.936896 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.936870    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:28.937175 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:28.937136    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac71ccb4_b649_43d7_a85c_bd783e16364e.slice/crio-d15ca20d16a164878f83a086cb61133a7743c4dfa033649b20259c4fdf540223 WatchSource:0}: Error finding container d15ca20d16a164878f83a086cb61133a7743c4dfa033649b20259c4fdf540223: Status 404 returned error can't find the container with id d15ca20d16a164878f83a086cb61133a7743c4dfa033649b20259c4fdf540223
Jul 14 11:18:28.946961 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.946828    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dgrpv"
Jul 14 11:18:28.947966 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:28.947895    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19adb3db_cb83_433e_bf71_997f74e2c60f.slice/crio-3dbb7a9264688920f2814daed2bd63df788863b1107b834cde8b7cebfa45aafb WatchSource:0}: Error finding container 3dbb7a9264688920f2814daed2bd63df788863b1107b834cde8b7cebfa45aafb: Status 404 returned error can't find the container with id 3dbb7a9264688920f2814daed2bd63df788863b1107b834cde8b7cebfa45aafb
Jul 14 11:18:28.956052 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.956019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5"
Jul 14 11:18:28.957036 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:18:28.957012    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531c861d_fdf2_4713_942d_c18700ba8aa5.slice/crio-df0156d48d7e6d4f347808c8e62ccc134c2f72ef408b4f583e6410c676153454 WatchSource:0}: Error finding container df0156d48d7e6d4f347808c8e62ccc134c2f72ef408b4f583e6410c676153454: Status 404 returned error can't find the container with id df0156d48d7e6d4f347808c8e62ccc134c2f72ef408b4f583e6410c676153454
Jul 14 11:18:28.963754 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:28.963727    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5mg9c"
Jul 14 11:18:29.191437 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.191341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:29.191609 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.191482    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:29.191609 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.191550    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret podName:e0ffcf77-e340-4c18-96d2-c069f045d964 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:30.191526987 +0000 UTC m=+3.992082726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret") pod "global-pull-secret-syncer-cbj6p" (UID: "e0ffcf77-e340-4c18-96d2-c069f045d964") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:29.292441 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.292394    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:29.292651 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.292589    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:29.292748 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.292677    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs podName:ab6dd4be-d083-4d37-b6b8-16abfcb09760 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:30.292654038 +0000 UTC m=+4.093209757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs") pod "network-metrics-daemon-89z9c" (UID: "ab6dd4be-d083-4d37-b6b8-16abfcb09760") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:29.393466 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.393424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:29.393641 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.393595    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:18:29.393641 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.393624    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:18:29.393641 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.393642    2564 projected.go:196] Error preparing data for projected volume kube-api-access-vpj89 for pod openshift-network-diagnostics/network-check-target-zp7vf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:29.393818 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.393709    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89 podName:fed8361b-4c48-4a30-8c36-8349ef0667f2 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:30.393689622 +0000 UTC m=+4.194245342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpj89" (UniqueName: "kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89") pod "network-check-target-zp7vf" (UID: "fed8361b-4c48-4a30-8c36-8349ef0667f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:29.628261 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.628207    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:13:28 +0000 UTC" deadline="2028-01-14 01:59:40.837010174 +0000 UTC"
Jul 14 11:18:29.628261 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.628251    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13166h41m11.208763086s"
Jul 14 11:18:29.765320 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.764569    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:29.765320 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.764711    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:29.765320 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.765159    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:29.765320 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:29.765261    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:29.775028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.774907    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"f70a6f929ea595658ac21b5a087363fff082c83e092b3ff7a18d8f91c42181cf"}
Jul 14 11:18:29.777712 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.777626    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bl2sn" event={"ID":"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998","Type":"ContainerStarted","Data":"2c50c166686ba8d98d1c8b47c0b1cb9d3d83ae08db78db1906768bd8ce1c1216"}
Jul 14 11:18:29.782965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.782927    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dgrpv" event={"ID":"531c861d-fdf2-4713-942d-c18700ba8aa5","Type":"ContainerStarted","Data":"df0156d48d7e6d4f347808c8e62ccc134c2f72ef408b4f583e6410c676153454"}
Jul 14 11:18:29.785111 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.785047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kl6lg" event={"ID":"19adb3db-cb83-433e-bf71-997f74e2c60f","Type":"ContainerStarted","Data":"3dbb7a9264688920f2814daed2bd63df788863b1107b834cde8b7cebfa45aafb"}
Jul 14 11:18:29.788314 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.788286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnrnz" event={"ID":"ac71ccb4-b649-43d7-a85c-bd783e16364e","Type":"ContainerStarted","Data":"d15ca20d16a164878f83a086cb61133a7743c4dfa033649b20259c4fdf540223"}
Jul 14 11:18:29.791047 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.790999    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jzknk" event={"ID":"46ec4cdd-1108-4609-b041-5140ab4972f4","Type":"ContainerStarted","Data":"7ef229f44b40ff9ef67cbe8295eee4651b649b483534eff1443a6eef95d440d1"}
Jul 14 11:18:29.793443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:29.793372    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerStarted","Data":"6ce365689c4be0098288b831f24e9fec0e66c3707f0783376ebe9590d0770584"}
Jul 14 11:18:30.201171 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.201129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:30.201384 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.201358    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:30.201532 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.201424    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret podName:e0ffcf77-e340-4c18-96d2-c069f045d964 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:32.201406459 +0000 UTC m=+6.001962184 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret") pod "global-pull-secret-syncer-cbj6p" (UID: "e0ffcf77-e340-4c18-96d2-c069f045d964") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:30.302350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.301677    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:30.302350 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.301899    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:30.302350 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.301965    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs podName:ab6dd4be-d083-4d37-b6b8-16abfcb09760 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:32.301945882 +0000 UTC m=+6.102501605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs") pod "network-metrics-daemon-89z9c" (UID: "ab6dd4be-d083-4d37-b6b8-16abfcb09760") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:30.402357 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.402315    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:30.402562 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.402523    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:18:30.402562 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.402548    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:18:30.402562 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.402562    2564 projected.go:196] Error preparing data for projected volume kube-api-access-vpj89 for pod openshift-network-diagnostics/network-check-target-zp7vf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:30.402740 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.402622    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89 podName:fed8361b-4c48-4a30-8c36-8349ef0667f2 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:32.402603058 +0000 UTC m=+6.203158785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpj89" (UniqueName: "kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89") pod "network-check-target-zp7vf" (UID: "fed8361b-4c48-4a30-8c36-8349ef0667f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:30.629036 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.628811    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:13:28 +0000 UTC" deadline="2028-03-14 07:28:12.823718511 +0000 UTC"
Jul 14 11:18:30.629164 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.629043    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14612h9m42.194685506s"
Jul 14 11:18:30.764024 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.763902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:30.764185 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:30.764066    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:30.804464 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.804339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5mg9c" event={"ID":"a6036834-7cc4-4b56-8044-4fc362b51b24","Type":"ContainerStarted","Data":"61af52a3f7c4ebcd3c5bedf19f8a01aabf36cdf4f46fb3b95bcf955a609978c3"}
Jul 14 11:18:30.812466 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.812370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5" event={"ID":"a523d557-2ff6-448e-a574-dd798eea5423","Type":"ContainerStarted","Data":"71781c029d367220f80473fd8d84154c9fd4f245eee2cceecac91012fc51cb9d"}
Jul 14 11:18:30.816659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:30.816630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal" event={"ID":"bdebd314971aba3dd18c52401ee8209e","Type":"ContainerStarted","Data":"16a36ff72a1675492f1905df146f0483f224984bbc3d75f4c4074d391dd146e1"}
Jul 14 11:18:31.763843 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:31.763804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:31.764036 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:31.763957    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:31.764388 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:31.764368    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:31.764486 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:31.764465    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:31.836910 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:31.836872    2564 generic.go:356] "Generic (PLEG): container finished" podID="6d16d29bd7cb1367aab205ff1ba54f40" containerID="d319e51a3fcd11dac12ebb2a5a59e79a6925ba7630a86ffd1afe761c5659a197" exitCode=0
Jul 14 11:18:31.840068 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:31.837894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal" event={"ID":"6d16d29bd7cb1367aab205ff1ba54f40","Type":"ContainerDied","Data":"d319e51a3fcd11dac12ebb2a5a59e79a6925ba7630a86ffd1afe761c5659a197"}
Jul 14 11:18:31.855277 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:31.855187    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-172.ec2.internal" podStartSLOduration=3.855166167 podStartE2EDuration="3.855166167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:30.832969694 +0000 UTC m=+4.633525440" watchObservedRunningTime="2026-07-14 11:18:31.855166167 +0000 UTC m=+5.655721911"
Jul 14 11:18:32.219110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:32.218334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:32.219110 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.218586    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:32.219110 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.218654    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret podName:e0ffcf77-e340-4c18-96d2-c069f045d964 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:36.218633769 +0000 UTC m=+10.019189504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret") pod "global-pull-secret-syncer-cbj6p" (UID: "e0ffcf77-e340-4c18-96d2-c069f045d964") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:32.320188 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:32.319554    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:32.320188 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.319759    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:32.320188 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.319832    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs podName:ab6dd4be-d083-4d37-b6b8-16abfcb09760 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:36.319809769 +0000 UTC m=+10.120365530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs") pod "network-metrics-daemon-89z9c" (UID: "ab6dd4be-d083-4d37-b6b8-16abfcb09760") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:32.420373 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:32.420331    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:32.420555 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.420512    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:18:32.420555 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.420545    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:18:32.420659 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.420560    2564 projected.go:196] Error preparing data for projected volume kube-api-access-vpj89 for pod openshift-network-diagnostics/network-check-target-zp7vf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:32.420659 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.420622    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89 podName:fed8361b-4c48-4a30-8c36-8349ef0667f2 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:36.420601864 +0000 UTC m=+10.221157608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpj89" (UniqueName: "kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89") pod "network-check-target-zp7vf" (UID: "fed8361b-4c48-4a30-8c36-8349ef0667f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:32.764998 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:32.764963    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:32.765178 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:32.765102    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:33.764290 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:33.763621    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:33.764290 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:33.763775    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:33.764290 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:33.764144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:33.764290 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:33.764238    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:34.763710 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:34.763535    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:34.763710 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:34.763684    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:35.763981 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:35.763935    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:35.764445 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:35.764077    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:35.764445 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:35.763952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:35.764583 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:35.764511    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:36.254386 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:36.254339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:36.254539 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.254494    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:36.254603 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.254576    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret podName:e0ffcf77-e340-4c18-96d2-c069f045d964 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:44.254552986 +0000 UTC m=+18.055108717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret") pod "global-pull-secret-syncer-cbj6p" (UID: "e0ffcf77-e340-4c18-96d2-c069f045d964") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:36.356253 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:36.355626    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:36.356253 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.355858    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:36.356253 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.355925    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs podName:ab6dd4be-d083-4d37-b6b8-16abfcb09760 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:44.355905647 +0000 UTC m=+18.156461378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs") pod "network-metrics-daemon-89z9c" (UID: "ab6dd4be-d083-4d37-b6b8-16abfcb09760") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:36.456374 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:36.456333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:36.456574 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.456512    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:18:36.456574 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.456533    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:18:36.456574 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.456546    2564 projected.go:196] Error preparing data for projected volume kube-api-access-vpj89 for pod openshift-network-diagnostics/network-check-target-zp7vf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:36.456969 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.456604    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89 podName:fed8361b-4c48-4a30-8c36-8349ef0667f2 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:44.456586318 +0000 UTC m=+18.257142036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpj89" (UniqueName: "kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89") pod "network-check-target-zp7vf" (UID: "fed8361b-4c48-4a30-8c36-8349ef0667f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:36.764784 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:36.764752    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:36.765308 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:36.764882    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:37.763580 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:37.763545    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:37.763752 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:37.763671    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:37.763864 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:37.763840    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:37.763994 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:37.763961    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:38.765676 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:38.765638    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:38.766122 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:38.765799    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:39.763990 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:39.763951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:39.763990 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:39.763992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:39.764190 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:39.764071    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:39.764242 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:39.764211    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:40.764040 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:40.764005    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:40.764521 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:40.764149    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:41.763446 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:41.763407    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:41.763619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:41.763407    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:41.763619 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:41.763551    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:41.763758 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:41.763616    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:42.764270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:42.764226    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:42.764735 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:42.764383    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:43.763742 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:43.763708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:43.763964 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:43.763711    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:43.763964 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:43.763824    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:43.763964 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:43.763907    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:44.315309 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:44.315267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:44.315685 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.315417    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:44.315685 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.315498    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret podName:e0ffcf77-e340-4c18-96d2-c069f045d964 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:00.315475505 +0000 UTC m=+34.116031259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret") pod "global-pull-secret-syncer-cbj6p" (UID: "e0ffcf77-e340-4c18-96d2-c069f045d964") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:18:44.415895 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:44.415861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:44.416062 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.415993    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:44.416125 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.416063    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs podName:ab6dd4be-d083-4d37-b6b8-16abfcb09760 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:00.416041706 +0000 UTC m=+34.216597424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs") pod "network-metrics-daemon-89z9c" (UID: "ab6dd4be-d083-4d37-b6b8-16abfcb09760") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:18:44.516294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:44.516260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:44.516467 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.516449    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:18:44.516513 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.516479    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:18:44.516513 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.516493    2564 projected.go:196] Error preparing data for projected volume kube-api-access-vpj89 for pod openshift-network-diagnostics/network-check-target-zp7vf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:44.516577 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.516558    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89 podName:fed8361b-4c48-4a30-8c36-8349ef0667f2 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:00.516536181 +0000 UTC m=+34.317091904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpj89" (UniqueName: "kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89") pod "network-check-target-zp7vf" (UID: "fed8361b-4c48-4a30-8c36-8349ef0667f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:18:44.764550 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:44.764468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:44.764713 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:44.764601    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:45.763790 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:45.763674    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:45.764191 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:45.763674    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:45.764191 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:45.763909    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:45.764191 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:45.763955    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:46.764915 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:46.764876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:46.765338 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:46.764992    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:47.763584 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:47.763550    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:47.763776 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:47.763685    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:47.763776 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:47.763738    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:47.763891 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:47.763859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:48.764667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.764175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:48.764667 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:48.764524    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:48.871877 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.871839    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5" event={"ID":"a523d557-2ff6-448e-a574-dd798eea5423","Type":"ContainerStarted","Data":"76b82b17e7182e3f99e768847641b2f28bf503c3aa81578d85c2c994805fc3d6"}
Jul 14 11:18:48.875377 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.875345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal" event={"ID":"6d16d29bd7cb1367aab205ff1ba54f40","Type":"ContainerStarted","Data":"227b58bb4a3e0b6f1718ccd1c49c2d64532b77c6d38c3c677fe011d0a3ee93c2"}
Jul 14 11:18:48.879635 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.878135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dgrpv" event={"ID":"531c861d-fdf2-4713-942d-c18700ba8aa5","Type":"ContainerStarted","Data":"c26ab99e43db4c64b8562dff5ec99e607928c1050bac92ea305543de96ba2652"}
Jul 14 11:18:48.880921 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.880891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kl6lg" event={"ID":"19adb3db-cb83-433e-bf71-997f74e2c60f","Type":"ContainerStarted","Data":"ffb239226997989419781a8ebd5d10c3f2b02d3d44b02a1a6ee5ba8902881a0e"}
Jul 14 11:18:48.882914 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.882891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnrnz" event={"ID":"ac71ccb4-b649-43d7-a85c-bd783e16364e","Type":"ContainerStarted","Data":"f144871ab32d202a83305e2958f88b554ce6153172c0788189620c6a79b6681c"}
Jul 14 11:18:48.885156 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.885125    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerStarted","Data":"4664efd9b6d9b9f14a1dde7da6311b834a83494fc02834d7355b5ba43aa246ee"}
Jul 14 11:18:48.891015 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.890952    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5mg9c" event={"ID":"a6036834-7cc4-4b56-8044-4fc362b51b24","Type":"ContainerStarted","Data":"0063725ca0c9069a44c1a627c88c4a36fefd755907498d09487de4c71e784185"}
Jul 14 11:18:48.893169 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.893130    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bl2sn" event={"ID":"c3ca13a8-f8ad-4565-b6f9-f3a5342e3998","Type":"ContainerStarted","Data":"febb821a2d7b543884e62a568cf61ed616349fcba4d6728a6597394316597a75"}
Jul 14 11:18:48.896144 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.896104    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-172.ec2.internal" podStartSLOduration=20.896091696 podStartE2EDuration="20.896091696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.895092915 +0000 UTC m=+22.695648658" watchObservedRunningTime="2026-07-14 11:18:48.896091696 +0000 UTC m=+22.696647437"
Jul 14 11:18:48.914571 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.914525    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bl2sn" podStartSLOduration=10.59513322 podStartE2EDuration="21.914508582s" totalImagesPullingTime="11.319375362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.914174371 +0000 UTC m=+22.714730112" watchObservedRunningTime="2026-07-14 11:18:48.914508582 +0000 UTC m=+22.715064321"
Jul 14 11:18:48.930992 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.930944    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5mg9c" podStartSLOduration=3.8806564359999998 podStartE2EDuration="21.930928312s" totalImagesPullingTime="18.050271876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.930540354 +0000 UTC m=+22.731096093" watchObservedRunningTime="2026-07-14 11:18:48.930928312 +0000 UTC m=+22.731484051"
Jul 14 11:18:48.980129 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:48.980070    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qnrnz" podStartSLOduration=2.322879513 podStartE2EDuration="21.980046951s" totalImagesPullingTime="19.657167438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.979619846 +0000 UTC m=+22.780175588" watchObservedRunningTime="2026-07-14 11:18:48.980046951 +0000 UTC m=+22.780602742"
Jul 14 11:18:49.023975 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.023933    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kl6lg" podStartSLOduration=2.3807717840000002 podStartE2EDuration="22.023915031s" totalImagesPullingTime="19.643143247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:49.003228005 +0000 UTC m=+22.803783744" watchObservedRunningTime="2026-07-14 11:18:49.023915031 +0000 UTC m=+22.824470772"
Jul 14 11:18:49.024428 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.024404    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dgrpv" podStartSLOduration=2.390242356 podStartE2EDuration="22.024394625s" totalImagesPullingTime="19.634152269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:49.023894444 +0000 UTC m=+22.824450202" watchObservedRunningTime="2026-07-14 11:18:49.024394625 +0000 UTC m=+22.824950364"
Jul 14 11:18:49.764227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.764018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:49.764428 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.764018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:49.764428 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:49.764390    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:49.764540 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:49.764479    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:49.896543 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.896471    2564 generic.go:356] "Generic (PLEG): container finished" podID="de80162a-fb30-416b-bda8-c41c436698b1" containerID="4664efd9b6d9b9f14a1dde7da6311b834a83494fc02834d7355b5ba43aa246ee" exitCode=0
Jul 14 11:18:49.897303 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.896554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerDied","Data":"4664efd9b6d9b9f14a1dde7da6311b834a83494fc02834d7355b5ba43aa246ee"}
Jul 14 11:18:49.899263 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.899242    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:18:49.899552 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.899533    2564 generic.go:356] "Generic (PLEG): container finished" podID="442f870e-3885-472d-8215-b2a5eeaa9a3a" containerID="ecf3f816ebb35d165b5845dfeaa52a625cff0af4e5f0afe727541fa1ce8c62f5" exitCode=1
Jul 14 11:18:49.899646 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.899626    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"57d7407b195d99133ba4326fdd2648f4465f7fa4bd76d929457ae92567e64aa3"}
Jul 14 11:18:49.899722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.899711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"a6d05f985834f908873c334915a97cda71a263e8a919878367c87ae4bdf10832"}
Jul 14 11:18:49.900205 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.900115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"f8850a29ed9b2c2608a83c9cabffc8ba75650d5e6e4fd2fe7f1e95f784e3e1e2"}
Jul 14 11:18:49.900269 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.900221    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"a3928e430b3761266bfdd966b39e26a825b97e5cc74e11d01651d46ec0604033"}
Jul 14 11:18:49.900323 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.900304    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerDied","Data":"ecf3f816ebb35d165b5845dfeaa52a625cff0af4e5f0afe727541fa1ce8c62f5"}
Jul 14 11:18:49.900599 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:49.900580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"3156f5010334cd7ca3986010ef7be313e916f8bd21ba8fe8c07e0f0cb8897fea"}
Jul 14 11:18:50.265853 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.265830    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 11:18:50.653419 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.653323    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T11:18:50.265847528Z","UUID":"f1739fc5-38fb-4779-b27f-96d9557674a0","Handler":null,"Name":"","Endpoint":""}
Jul 14 11:18:50.654944 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.654920    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 11:18:50.655070 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.654967    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 11:18:50.764401 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.764363    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:50.764583 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:50.764510    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:50.904597 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.904497    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jzknk" event={"ID":"46ec4cdd-1108-4609-b041-5140ab4972f4","Type":"ContainerStarted","Data":"93ee5c85c2294b8d48943f3dc121b1f1335daf4f2ba7f40012597a9b7cf7b687"}
Jul 14 11:18:50.907139 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.907072    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5" event={"ID":"a523d557-2ff6-448e-a574-dd798eea5423","Type":"ContainerStarted","Data":"46fba0f9de0f4be9f7a8d3ee04d3dce0e6a11be72128d45ee359990b2c64fd70"}
Jul 14 11:18:50.920818 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:50.920760    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jzknk" podStartSLOduration=4.247998362 podStartE2EDuration="23.920744285s" totalImagesPullingTime="19.672745923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:50.92001324 +0000 UTC m=+24.720568982" watchObservedRunningTime="2026-07-14 11:18:50.920744285 +0000 UTC m=+24.721300024"
Jul 14 11:18:51.764370 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:51.764079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:51.764538 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:51.764079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:51.764538 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:51.764438    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:51.764538 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:51.764512    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:51.913092 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:51.913057    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:18:51.913527 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:51.913508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"13c307198e08460245e956fa56d8fb3573e10f23c3a584a3ab6f16f0f15b5187"}
Jul 14 11:18:51.916007 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:51.915978    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5" event={"ID":"a523d557-2ff6-448e-a574-dd798eea5423","Type":"ContainerStarted","Data":"1326145927e67e26698d97cef2ebe98c5649d1f7e2466d5e02b0ccf7e5d73ccb"}
Jul 14 11:18:51.935656 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:51.935600    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h6bg5" podStartSLOduration=4.433663445 podStartE2EDuration="24.935580904s" totalImagesPullingTime="20.501917459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:51.935217608 +0000 UTC m=+25.735773347" watchObservedRunningTime="2026-07-14 11:18:51.935580904 +0000 UTC m=+25.736136645"
Jul 14 11:18:52.764256 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:52.764222    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:52.764442 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:52.764368    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:52.901717 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:52.901634    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:52.902453 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:52.902433    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:52.919244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:52.919216    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:52.919859 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:52.919426    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kl6lg"
Jul 14 11:18:53.764233 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:53.764195    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:53.764233 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:53.764241    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:53.764461 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:53.764320    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:53.764461 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:53.764415    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:54.763754 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:54.763484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:54.764283 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:54.763878    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:54.801119 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:54.801059    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dgrpv_531c861d-fdf2-4713-942d-c18700ba8aa5/dns-node-resolver/0.log"
Jul 14 11:18:54.924171 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:54.924138    2564 generic.go:356] "Generic (PLEG): container finished" podID="de80162a-fb30-416b-bda8-c41c436698b1" containerID="530c06214a91a7bd974fdd69b7bc364c0631d94d38418a291eff3231ad542007" exitCode=0
Jul 14 11:18:54.924334 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:54.924215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerDied","Data":"530c06214a91a7bd974fdd69b7bc364c0631d94d38418a291eff3231ad542007"}
Jul 14 11:18:54.929782 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:54.929734    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:18:54.930160 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:54.930128    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"f2a9af0d8a619b84cb1600fb36aa764e712b11dc76f4e73de57f537c994d014f"}
Jul 14 11:18:54.930599 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:54.930583    2564 scope.go:117] "RemoveContainer" containerID="ecf3f816ebb35d165b5845dfeaa52a625cff0af4e5f0afe727541fa1ce8c62f5"
Jul 14 11:18:55.764547 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.764389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:55.765024 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.764439    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:55.765024 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:55.764621    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:55.765024 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:55.764746    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:55.776596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.776578    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bl2sn_c3ca13a8-f8ad-4565-b6f9-f3a5342e3998/node-ca/0.log"
Jul 14 11:18:55.884368 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.884340    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zp7vf"]
Jul 14 11:18:55.885424 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.885388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbj6p"]
Jul 14 11:18:55.885908 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.885867    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-89z9c"]
Jul 14 11:18:55.886005 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.885985    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:55.886151 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:55.886128    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:55.935769 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.935751    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:18:55.936164 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.936146    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:55.936252 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.936177    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" event={"ID":"442f870e-3885-472d-8215-b2a5eeaa9a3a","Type":"ContainerStarted","Data":"5956f9422c38f0a295d6da3d45468e79ba7942316feb80960b3c4b150cb30b9e"}
Jul 14 11:18:55.936357 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:55.936264    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:55.936467 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.936450    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:55.936528 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.936474    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:55.936713 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.936551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:55.936713 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:55.936551    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:55.936713 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.936604    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:55.952362 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.952343    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:55.954407 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.954387    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:18:55.966165 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:55.966117    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4" podStartSLOduration=9.088924151 podStartE2EDuration="28.966103254s" totalImagesPullingTime="19.877179103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:55.964807705 +0000 UTC m=+29.765363444" watchObservedRunningTime="2026-07-14 11:18:55.966103254 +0000 UTC m=+29.766658994"
Jul 14 11:18:56.940261 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:56.940219    2564 generic.go:356] "Generic (PLEG): container finished" podID="de80162a-fb30-416b-bda8-c41c436698b1" containerID="b8a5507e7e733855656f702e8034e902f9895b1723952e1caa4bec0fba0282b2" exitCode=0
Jul 14 11:18:56.940822 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:56.940298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerDied","Data":"b8a5507e7e733855656f702e8034e902f9895b1723952e1caa4bec0fba0282b2"}
Jul 14 11:18:57.764438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:57.764404    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:57.764438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:57.764432    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:57.764605 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:57.764519    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:18:57.764605 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:57.764577    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:57.764749 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:57.764725    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:57.764817 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:57.764788    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:57.946356 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:57.946328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerStarted","Data":"57739dce9b5172e2142db4271542475d7585673d2176fb6f2a780e418f3a225a"}
Jul 14 11:18:58.950271 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:58.950239    2564 generic.go:356] "Generic (PLEG): container finished" podID="de80162a-fb30-416b-bda8-c41c436698b1" containerID="57739dce9b5172e2142db4271542475d7585673d2176fb6f2a780e418f3a225a" exitCode=0
Jul 14 11:18:58.950822 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:58.950300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerDied","Data":"57739dce9b5172e2142db4271542475d7585673d2176fb6f2a780e418f3a225a"}
Jul 14 11:18:59.764497 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:59.764412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:18:59.764662 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:59.764412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:18:59.764662 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:59.764534    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-89z9c" podUID="ab6dd4be-d083-4d37-b6b8-16abfcb09760"
Jul 14 11:18:59.764662 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:18:59.764412    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:18:59.764662 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:59.764605    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zp7vf" podUID="fed8361b-4c48-4a30-8c36-8349ef0667f2"
Jul 14 11:18:59.764861 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:18:59.764726    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cbj6p" podUID="e0ffcf77-e340-4c18-96d2-c069f045d964"
Jul 14 11:19:00.340608 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:00.340565    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:19:00.340974 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.340767    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 11:19:00.340974 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.340837    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret podName:e0ffcf77-e340-4c18-96d2-c069f045d964 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:32.340817656 +0000 UTC m=+66.141373392 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret") pod "global-pull-secret-syncer-cbj6p" (UID: "e0ffcf77-e340-4c18-96d2-c069f045d964") : object "kube-system"/"original-pull-secret" not registered
Jul 14 11:19:00.441821 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:00.441786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:19:00.442003 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.441949    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:19:00.442070 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.442013    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs podName:ab6dd4be-d083-4d37-b6b8-16abfcb09760 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:32.441996993 +0000 UTC m=+66.242552716 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs") pod "network-metrics-daemon-89z9c" (UID: "ab6dd4be-d083-4d37-b6b8-16abfcb09760") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 11:19:00.542534 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:00.542496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:19:00.542728 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.542663    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 11:19:00.542728 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.542688    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 11:19:00.542728 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.542711    2564 projected.go:196] Error preparing data for projected volume kube-api-access-vpj89 for pod openshift-network-diagnostics/network-check-target-zp7vf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:19:00.542900 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:00.542768    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89 podName:fed8361b-4c48-4a30-8c36-8349ef0667f2 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:32.542748125 +0000 UTC m=+66.343303863 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpj89" (UniqueName: "kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89") pod "network-check-target-zp7vf" (UID: "fed8361b-4c48-4a30-8c36-8349ef0667f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 11:19:01.517917 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.517734    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-172.ec2.internal" event="NodeReady"
Jul 14 11:19:01.518299 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.518042    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 11:19:01.563020 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.562990    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z5f9m"]
Jul 14 11:19:01.565915 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.565891    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q78dz"]
Jul 14 11:19:01.566081 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.566000    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.568016 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.567992    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.568630 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.568300    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 11:19:01.568630 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.568417    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 11:19:01.568814 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.568729    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgqwk\""
Jul 14 11:19:01.569938 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.569920    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 11:19:01.570064 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.570038    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 11:19:01.570191 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.570174    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 11:19:01.570547 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.570527    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 11:19:01.570547 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.570531    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xrcwt\""
Jul 14 11:19:01.577271 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.577216    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z5f9m"]
Jul 14 11:19:01.580757 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.580690    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q78dz"]
Jul 14 11:19:01.587071 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.587049    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5wpn6"]
Jul 14 11:19:01.588851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.588834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.591645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.591627    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9c655\""
Jul 14 11:19:01.592397 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.592380    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 11:19:01.592951 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.592932    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 11:19:01.594047 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.594031    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 11:19:01.599094 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.599075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5wpn6"]
Jul 14 11:19:01.754178 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754145    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbbmh\" (UniqueName: \"kubernetes.io/projected/813f9693-f3e7-4c38-8ed1-7be66382f4ef-kube-api-access-kbbmh\") pod \"ingress-canary-5wpn6\" (UID: \"813f9693-f3e7-4c38-8ed1-7be66382f4ef\") " pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.754366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st76g\" (UniqueName: \"kubernetes.io/projected/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-api-access-st76g\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.754366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754240    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/813f9693-f3e7-4c38-8ed1-7be66382f4ef-cert\") pod \"ingress-canary-5wpn6\" (UID: \"813f9693-f3e7-4c38-8ed1-7be66382f4ef\") " pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.754366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754271    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-tmp-dir\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.754366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-crio-socket\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.754366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754314    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-data-volume\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.754366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.754619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754371    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.754619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cb8f\" (UniqueName: \"kubernetes.io/projected/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-kube-api-access-8cb8f\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.754619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.754619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.754530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-config-volume\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.763431 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.763401    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:19:01.763570 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.763401    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:19:01.763628 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.763401    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:19:01.766382 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.766272    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 11:19:01.767214 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.766427    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 11:19:01.767214 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.766487    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 11:19:01.767214 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.766509    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 11:19:01.767214 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.766532    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q2rq4\""
Jul 14 11:19:01.767214 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.766792    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bm9fn\""
Jul 14 11:19:01.855094 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855055    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.855294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-config-volume\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.855294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbbmh\" (UniqueName: \"kubernetes.io/projected/813f9693-f3e7-4c38-8ed1-7be66382f4ef-kube-api-access-kbbmh\") pod \"ingress-canary-5wpn6\" (UID: \"813f9693-f3e7-4c38-8ed1-7be66382f4ef\") " pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.855294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st76g\" (UniqueName: \"kubernetes.io/projected/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-api-access-st76g\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.855294 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:01.855222    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:01.855294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855241    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/813f9693-f3e7-4c38-8ed1-7be66382f4ef-cert\") pod \"ingress-canary-5wpn6\" (UID: \"813f9693-f3e7-4c38-8ed1-7be66382f4ef\") " pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.855294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-tmp-dir\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.855294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-crio-socket\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:01.855313    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.355291331 +0000 UTC m=+36.155847067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-data-volume\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855428    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cb8f\" (UniqueName: \"kubernetes.io/projected/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-kube-api-access-8cb8f\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-crio-socket\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-tmp-dir\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.855614 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:01.855602    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:19:01.856056 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:01.855665    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.35564488 +0000 UTC m=+36.156200616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : configmap "kube-rbac-proxy" not found
Jul 14 11:19:01.856056 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:01.855748    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:19:01.856056 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:01.855803    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls podName:aeadd9fa-ef3b-40f2-b2f7-e69191293b3b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.355787512 +0000 UTC m=+36.156343248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls") pod "dns-default-z5f9m" (UID: "aeadd9fa-ef3b-40f2-b2f7-e69191293b3b") : secret "dns-default-metrics-tls" not found
Jul 14 11:19:01.856056 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-config-volume\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.856056 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.855808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-data-volume\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.859779 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.859758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/813f9693-f3e7-4c38-8ed1-7be66382f4ef-cert\") pod \"ingress-canary-5wpn6\" (UID: \"813f9693-f3e7-4c38-8ed1-7be66382f4ef\") " pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.869386 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.869359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cb8f\" (UniqueName: \"kubernetes.io/projected/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-kube-api-access-8cb8f\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:01.870269 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.870243    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st76g\" (UniqueName: \"kubernetes.io/projected/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-api-access-st76g\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:01.870800 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.870776    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbbmh\" (UniqueName: \"kubernetes.io/projected/813f9693-f3e7-4c38-8ed1-7be66382f4ef-kube-api-access-kbbmh\") pod \"ingress-canary-5wpn6\" (UID: \"813f9693-f3e7-4c38-8ed1-7be66382f4ef\") " pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.888873 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.888847    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-4c55w"]
Jul 14 11:19:01.891545 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.891523    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:01.894489 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.894464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 14 11:19:01.894659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.894636    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zbnxr\""
Jul 14 11:19:01.895062 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.895040    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 14 11:19:01.895172 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.895082    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 14 11:19:01.895172 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.895040    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 14 11:19:01.898408 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.898388    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5wpn6"
Jul 14 11:19:01.904398 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:01.904377    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-4c55w"]
Jul 14 11:19:02.057272 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.057186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/791b909d-7ee9-4a19-bfba-bccbe457eca7-signing-key\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.057272 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.057230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/791b909d-7ee9-4a19-bfba-bccbe457eca7-signing-cabundle\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.057480 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.057309    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5lrw\" (UniqueName: \"kubernetes.io/projected/791b909d-7ee9-4a19-bfba-bccbe457eca7-kube-api-access-p5lrw\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.158002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.157967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5lrw\" (UniqueName: \"kubernetes.io/projected/791b909d-7ee9-4a19-bfba-bccbe457eca7-kube-api-access-p5lrw\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.158170 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.158046    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/791b909d-7ee9-4a19-bfba-bccbe457eca7-signing-key\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.158170 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.158074    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/791b909d-7ee9-4a19-bfba-bccbe457eca7-signing-cabundle\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.158809 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.158771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/791b909d-7ee9-4a19-bfba-bccbe457eca7-signing-cabundle\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.160730 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.160683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/791b909d-7ee9-4a19-bfba-bccbe457eca7-signing-key\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.167445 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.167422    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5lrw\" (UniqueName: \"kubernetes.io/projected/791b909d-7ee9-4a19-bfba-bccbe457eca7-kube-api-access-p5lrw\") pod \"service-ca-79f68b9f4f-4c55w\" (UID: \"791b909d-7ee9-4a19-bfba-bccbe457eca7\") " pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.202043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.202011    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w"
Jul 14 11:19:02.359924 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.359837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:02.359924 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.359901    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:02.359924 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:02.359923    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:19:02.360177 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.359939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:02.360177 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:02.359987    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:03.359970007 +0000 UTC m=+37.160525746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : configmap "kube-rbac-proxy" not found
Jul 14 11:19:02.360177 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:02.360053    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:02.360177 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:02.360071    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:19:02.360177 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:02.360098    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:03.360085698 +0000 UTC m=+37.160641434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:02.360177 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:02.360132    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls podName:aeadd9fa-ef3b-40f2-b2f7-e69191293b3b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:03.36011454 +0000 UTC m=+37.160670281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls") pod "dns-default-z5f9m" (UID: "aeadd9fa-ef3b-40f2-b2f7-e69191293b3b") : secret "dns-default-metrics-tls" not found
Jul 14 11:19:02.970410 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.970384    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-4c55w"]
Jul 14 11:19:02.974598 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:02.974410    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5wpn6"]
Jul 14 11:19:02.975042 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:02.975013    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod791b909d_7ee9_4a19_bfba_bccbe457eca7.slice/crio-100c973e86000c3e5546a740403cd8ace91f7df38edc23944dd02e2b1a344050 WatchSource:0}: Error finding container 100c973e86000c3e5546a740403cd8ace91f7df38edc23944dd02e2b1a344050: Status 404 returned error can't find the container with id 100c973e86000c3e5546a740403cd8ace91f7df38edc23944dd02e2b1a344050
Jul 14 11:19:02.977788 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:02.977763    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813f9693_f3e7_4c38_8ed1_7be66382f4ef.slice/crio-25c0595d6903ae614a99571cbbaa0d5e2390266c345d9a1f9447c645239a4fdd WatchSource:0}: Error finding container 25c0595d6903ae614a99571cbbaa0d5e2390266c345d9a1f9447c645239a4fdd: Status 404 returned error can't find the container with id 25c0595d6903ae614a99571cbbaa0d5e2390266c345d9a1f9447c645239a4fdd
Jul 14 11:19:03.375168 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:03.375062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:03.375168 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:03.375113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:03.375415 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:03.375190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:03.375415 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:03.375236    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:19:03.375415 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:03.375289    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls podName:aeadd9fa-ef3b-40f2-b2f7-e69191293b3b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:05.375275739 +0000 UTC m=+39.175831470 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls") pod "dns-default-z5f9m" (UID: "aeadd9fa-ef3b-40f2-b2f7-e69191293b3b") : secret "dns-default-metrics-tls" not found
Jul 14 11:19:03.375415 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:03.375321    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:03.375415 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:03.375347    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:19:03.375415 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:03.375393    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:05.375374828 +0000 UTC m=+39.175930565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:03.375415 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:03.375416    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:05.37540548 +0000 UTC m=+39.175961203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : configmap "kube-rbac-proxy" not found
Jul 14 11:19:03.966863 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:03.966644    2564 generic.go:356] "Generic (PLEG): container finished" podID="de80162a-fb30-416b-bda8-c41c436698b1" containerID="c0b2b25108ebc8bcc8994ef414fe379e97e293589d3f876737e604ae42bd4bff" exitCode=0
Jul 14 11:19:03.966863 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:03.966790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerDied","Data":"c0b2b25108ebc8bcc8994ef414fe379e97e293589d3f876737e604ae42bd4bff"}
Jul 14 11:19:03.969107 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:03.969039    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5wpn6" event={"ID":"813f9693-f3e7-4c38-8ed1-7be66382f4ef","Type":"ContainerStarted","Data":"25c0595d6903ae614a99571cbbaa0d5e2390266c345d9a1f9447c645239a4fdd"}
Jul 14 11:19:03.970597 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:03.970564    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w" event={"ID":"791b909d-7ee9-4a19-bfba-bccbe457eca7","Type":"ContainerStarted","Data":"100c973e86000c3e5546a740403cd8ace91f7df38edc23944dd02e2b1a344050"}
Jul 14 11:19:04.976726 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:04.976674    2564 generic.go:356] "Generic (PLEG): container finished" podID="de80162a-fb30-416b-bda8-c41c436698b1" containerID="599999a8ab58e9b4c93405e59cce5d568cd41c92290a0e5e0863cbc16f570e00" exitCode=0
Jul 14 11:19:04.977336 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:04.976760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerDied","Data":"599999a8ab58e9b4c93405e59cce5d568cd41c92290a0e5e0863cbc16f570e00"}
Jul 14 11:19:05.390669 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:05.390628    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:05.390886 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:05.390684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:05.390886 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:05.390787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:05.390886 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:05.390842    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:19:05.390886 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:05.390870    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 11:19:05.391098 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:05.390895    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:05.391098 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:05.390949    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls podName:aeadd9fa-ef3b-40f2-b2f7-e69191293b3b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:09.39092432 +0000 UTC m=+43.191480061 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls") pod "dns-default-z5f9m" (UID: "aeadd9fa-ef3b-40f2-b2f7-e69191293b3b") : secret "dns-default-metrics-tls" not found
Jul 14 11:19:05.391098 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:05.390972    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:09.390960822 +0000 UTC m=+43.191516549 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:05.391098 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:05.390991    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:09.390982422 +0000 UTC m=+43.191538142 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : configmap "kube-rbac-proxy" not found
Jul 14 11:19:05.981381 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:05.981347    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w" event={"ID":"791b909d-7ee9-4a19-bfba-bccbe457eca7","Type":"ContainerStarted","Data":"5686cb1c26c57057069097e5845f20152aa7909b53451199954368117729c8c9"}
Jul 14 11:19:05.985514 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:05.985483    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" event={"ID":"de80162a-fb30-416b-bda8-c41c436698b1","Type":"ContainerStarted","Data":"bf9627a7c45e5cc385df3f9b16b06086fdffb75bddd3ffdfc6ce409e8ae7a21c"}
Jul 14 11:19:05.986991 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:05.986969    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5wpn6" event={"ID":"813f9693-f3e7-4c38-8ed1-7be66382f4ef","Type":"ContainerStarted","Data":"a9d571fdd6e90ff3b102eb1bb9ea3cd6eaba756e0af58c8cf1dce54b6fe7565b"}
Jul 14 11:19:05.998637 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:05.998589    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-4c55w" podStartSLOduration=2.167948001 podStartE2EDuration="4.998574029s" totalImagesPullingTime="2.830626028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:05.997740055 +0000 UTC m=+39.798295795" watchObservedRunningTime="2026-07-14 11:19:05.998574029 +0000 UTC m=+39.799129768"
Jul 14 11:19:06.013438 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:06.013388    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5wpn6" podStartSLOduration=2.192008833 podStartE2EDuration="5.013368604s" totalImagesPullingTime="2.821359771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:06.01271551 +0000 UTC m=+39.813271241" watchObservedRunningTime="2026-07-14 11:19:06.013368604 +0000 UTC m=+39.813924326"
Jul 14 11:19:06.037325 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:06.037227    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l5sjl" podStartSLOduration=9.614413371 podStartE2EDuration="39.03721153s" totalImagesPullingTime="29.422798159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:06.035886702 +0000 UTC m=+39.836442443" watchObservedRunningTime="2026-07-14 11:19:06.03721153 +0000 UTC m=+39.837767269"
Jul 14 11:19:09.421906 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:09.421642    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:09.422326 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:09.421945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:09.422326 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:09.421990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:09.422326 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:09.422119    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 11:19:09.422326 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:09.422146    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:09.422326 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:09.422199    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls podName:aeadd9fa-ef3b-40f2-b2f7-e69191293b3b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:17.42217824 +0000 UTC m=+51.222733965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls") pod "dns-default-z5f9m" (UID: "aeadd9fa-ef3b-40f2-b2f7-e69191293b3b") : secret "dns-default-metrics-tls" not found
Jul 14 11:19:09.422326 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:09.422217    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls podName:08dd0d2a-d514-4f3b-b980-b8dfdf55e50f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:17.422208131 +0000 UTC m=+51.222763853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q78dz" (UID: "08dd0d2a-d514-4f3b-b980-b8dfdf55e50f") : secret "insights-runtime-extractor-tls" not found
Jul 14 11:19:09.422326 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:09.422298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:17.482458 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.482420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:17.482947 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.482465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:17.486150 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.486111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeadd9fa-ef3b-40f2-b2f7-e69191293b3b-metrics-tls\") pod \"dns-default-z5f9m\" (UID: \"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b\") " pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:17.486150 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.486122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/08dd0d2a-d514-4f3b-b980-b8dfdf55e50f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q78dz\" (UID: \"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f\") " pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:17.778845 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.778819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:17.786029 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.786007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q78dz"
Jul 14 11:19:17.921240 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.921209    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z5f9m"]
Jul 14 11:19:17.923889 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:17.923861    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeadd9fa_ef3b_40f2_b2f7_e69191293b3b.slice/crio-4ead5e6c9a940dfc2f2a79ad3c6a5c4b9118723de86fb19d88e4b4a815dafb43 WatchSource:0}: Error finding container 4ead5e6c9a940dfc2f2a79ad3c6a5c4b9118723de86fb19d88e4b4a815dafb43: Status 404 returned error can't find the container with id 4ead5e6c9a940dfc2f2a79ad3c6a5c4b9118723de86fb19d88e4b4a815dafb43
Jul 14 11:19:17.937328 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:17.937306    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q78dz"]
Jul 14 11:19:17.940087 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:17.940067    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dd0d2a_d514_4f3b_b980_b8dfdf55e50f.slice/crio-2c2986a64be91a1a844622523b4326a2273134a19aeeeeabe0061ef21ff23f3f WatchSource:0}: Error finding container 2c2986a64be91a1a844622523b4326a2273134a19aeeeeabe0061ef21ff23f3f: Status 404 returned error can't find the container with id 2c2986a64be91a1a844622523b4326a2273134a19aeeeeabe0061ef21ff23f3f
Jul 14 11:19:18.020843 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:18.020806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q78dz" event={"ID":"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f","Type":"ContainerStarted","Data":"2c2986a64be91a1a844622523b4326a2273134a19aeeeeabe0061ef21ff23f3f"}
Jul 14 11:19:18.021908 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:18.021886    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z5f9m" event={"ID":"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b","Type":"ContainerStarted","Data":"4ead5e6c9a940dfc2f2a79ad3c6a5c4b9118723de86fb19d88e4b4a815dafb43"}
Jul 14 11:19:22.163998 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.163959    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b66646578-mlbpl"]
Jul 14 11:19:22.166148 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.166127    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.170396 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.170373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 14 11:19:22.170527 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.170412    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 14 11:19:22.170687 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.170673    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rhbkm\""
Jul 14 11:19:22.171006 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.170989    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 14 11:19:22.181410 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.181389    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 14 11:19:22.188406 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.188379    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b66646578-mlbpl"]
Jul 14 11:19:22.220295 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220209    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85e379da-4380-4406-9f31-f9730d9ffd46-ca-trust-extracted\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.220295 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220291    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85e379da-4380-4406-9f31-f9730d9ffd46-trusted-ca\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.220467 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85e379da-4380-4406-9f31-f9730d9ffd46-image-registry-private-configuration\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.220467 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220432    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85e379da-4380-4406-9f31-f9730d9ffd46-installation-pull-secrets\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.220582 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-registry-tls\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.220582 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220512    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-bound-sa-token\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.220582 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85e379da-4380-4406-9f31-f9730d9ffd46-registry-certificates\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.220582 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.220565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k295n\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-kube-api-access-k295n\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.320961 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.320928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85e379da-4380-4406-9f31-f9730d9ffd46-ca-trust-extracted\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321119 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.320990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85e379da-4380-4406-9f31-f9730d9ffd46-trusted-ca\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321191 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.321137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85e379da-4380-4406-9f31-f9730d9ffd46-image-registry-private-configuration\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321246 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.321220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85e379da-4380-4406-9f31-f9730d9ffd46-installation-pull-secrets\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321295 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.321270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-registry-tls\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321420 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.321395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-bound-sa-token\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321420 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.321410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/85e379da-4380-4406-9f31-f9730d9ffd46-ca-trust-extracted\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321543 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.321523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85e379da-4380-4406-9f31-f9730d9ffd46-registry-certificates\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.321599 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.321563    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k295n\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-kube-api-access-k295n\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.322115 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.322091    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85e379da-4380-4406-9f31-f9730d9ffd46-trusted-ca\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.322327 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.322309    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/85e379da-4380-4406-9f31-f9730d9ffd46-registry-certificates\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.323722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.323688    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-registry-tls\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.323722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.323690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/85e379da-4380-4406-9f31-f9730d9ffd46-installation-pull-secrets\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.323970 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.323953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/85e379da-4380-4406-9f31-f9730d9ffd46-image-registry-private-configuration\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.331530 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.331504    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k295n\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-kube-api-access-k295n\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.331851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.331827    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85e379da-4380-4406-9f31-f9730d9ffd46-bound-sa-token\") pod \"image-registry-6b66646578-mlbpl\" (UID: \"85e379da-4380-4406-9f31-f9730d9ffd46\") " pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.477274 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.477190    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:22.616984 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:22.616951    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b66646578-mlbpl"]
Jul 14 11:19:22.619987 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:22.619956    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e379da_4380_4406_9f31_f9730d9ffd46.slice/crio-1eeea9dfa47ee9dc7b0b02d79554829c9976007a587614dc6fbd235b6ddca694 WatchSource:0}: Error finding container 1eeea9dfa47ee9dc7b0b02d79554829c9976007a587614dc6fbd235b6ddca694: Status 404 returned error can't find the container with id 1eeea9dfa47ee9dc7b0b02d79554829c9976007a587614dc6fbd235b6ddca694
Jul 14 11:19:23.040336 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.040295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z5f9m" event={"ID":"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b","Type":"ContainerStarted","Data":"d15361d4d79e4d62740a774bab658fe5280e6c2eae012e2b6f8f460971d1f707"}
Jul 14 11:19:23.040556 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.040395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z5f9m" event={"ID":"aeadd9fa-ef3b-40f2-b2f7-e69191293b3b","Type":"ContainerStarted","Data":"1a454b992afbae1758ceb962620ebc25e36f54b920d27a5fa29bfc3a2f14af5b"}
Jul 14 11:19:23.040556 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.040431    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:23.042187 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.042159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q78dz" event={"ID":"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f","Type":"ContainerStarted","Data":"b182841ece4d320d865bd4ce54c99d260c4cab0a2363d1d3c8ccb476fc55acf5"}
Jul 14 11:19:23.042310 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.042211    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q78dz" event={"ID":"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f","Type":"ContainerStarted","Data":"a401802dd6a3195719493a5661ea0bd4865e4f17ad40ff11b56ed727c284de27"}
Jul 14 11:19:23.043664 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.043635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b66646578-mlbpl" event={"ID":"85e379da-4380-4406-9f31-f9730d9ffd46","Type":"ContainerStarted","Data":"eb05a75d80ecdb00b35568aae2266a4dd734d4f09cf60e70bf593617abee840c"}
Jul 14 11:19:23.043775 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.043683    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b66646578-mlbpl" event={"ID":"85e379da-4380-4406-9f31-f9730d9ffd46","Type":"ContainerStarted","Data":"1eeea9dfa47ee9dc7b0b02d79554829c9976007a587614dc6fbd235b6ddca694"}
Jul 14 11:19:23.043775 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.043743    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:23.089859 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.089799    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z5f9m" podStartSLOduration=17.847160437 podStartE2EDuration="22.089777205s" totalImagesPullingTime="4.242616768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:23.088491516 +0000 UTC m=+56.889047268" watchObservedRunningTime="2026-07-14 11:19:23.089777205 +0000 UTC m=+56.890332946"
Jul 14 11:19:23.113808 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:23.113749    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b66646578-mlbpl" podStartSLOduration=1.113733765 podStartE2EDuration="1.113733765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:23.112933962 +0000 UTC m=+56.913489704" watchObservedRunningTime="2026-07-14 11:19:23.113733765 +0000 UTC m=+56.914289504"
Jul 14 11:19:24.033471 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.033435    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"]
Jul 14 11:19:24.036141 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.036118    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"
Jul 14 11:19:24.038783 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.038761    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bd49d\""
Jul 14 11:19:24.038903 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.038847    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 14 11:19:24.044942 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.044917    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"]
Jul 14 11:19:24.135618 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.135581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/67309b0e-d8d1-4b17-83f9-d7712eede2b0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7p5z4\" (UID: \"67309b0e-d8d1-4b17-83f9-d7712eede2b0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"
Jul 14 11:19:24.236496 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.236453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/67309b0e-d8d1-4b17-83f9-d7712eede2b0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7p5z4\" (UID: \"67309b0e-d8d1-4b17-83f9-d7712eede2b0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"
Jul 14 11:19:24.239035 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.239012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/67309b0e-d8d1-4b17-83f9-d7712eede2b0-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7p5z4\" (UID: \"67309b0e-d8d1-4b17-83f9-d7712eede2b0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"
Jul 14 11:19:24.347305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:24.347218    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"
Jul 14 11:19:25.235998 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:25.235970    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"]
Jul 14 11:19:25.238932 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:25.238909    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67309b0e_d8d1_4b17_83f9_d7712eede2b0.slice/crio-a20ffbd2445b2ee0fb8e2ec665c184f2f59714112816812ba48bc68c1da5ae08 WatchSource:0}: Error finding container a20ffbd2445b2ee0fb8e2ec665c184f2f59714112816812ba48bc68c1da5ae08: Status 404 returned error can't find the container with id a20ffbd2445b2ee0fb8e2ec665c184f2f59714112816812ba48bc68c1da5ae08
Jul 14 11:19:26.061554 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:26.061511    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q78dz" event={"ID":"08dd0d2a-d514-4f3b-b980-b8dfdf55e50f","Type":"ContainerStarted","Data":"2fa8a383b61c9750b988bff818cfa4346a8aaa3ec286827ff49ad98bb2b1e5ff"}
Jul 14 11:19:26.062657 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:26.062631    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4" event={"ID":"67309b0e-d8d1-4b17-83f9-d7712eede2b0","Type":"ContainerStarted","Data":"a20ffbd2445b2ee0fb8e2ec665c184f2f59714112816812ba48bc68c1da5ae08"}
Jul 14 11:19:26.082217 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:26.082161    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q78dz" podStartSLOduration=17.992668299 podStartE2EDuration="25.082146779s" totalImagesPullingTime="7.08947848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:19:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:26.081040104 +0000 UTC m=+59.881595877" watchObservedRunningTime="2026-07-14 11:19:26.082146779 +0000 UTC m=+59.882702518"
Jul 14 11:19:27.067674 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:27.067625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4" event={"ID":"67309b0e-d8d1-4b17-83f9-d7712eede2b0","Type":"ContainerStarted","Data":"589176f88a31d7cc5fcc65b0dfbdf28de6cafa52550475c28fbe70ee26ea7f9e"}
Jul 14 11:19:27.068160 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:27.067715    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"
Jul 14 11:19:27.072384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:27.072361    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4"
Jul 14 11:19:27.085138 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:27.085094    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7p5z4" podStartSLOduration=1.930215934 podStartE2EDuration="3.085081083s" totalImagesPullingTime="1.154865149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:27.084266495 +0000 UTC m=+60.884822232" watchObservedRunningTime="2026-07-14 11:19:27.085081083 +0000 UTC m=+60.885636823"
Jul 14 11:19:27.960149 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:27.960116    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d5lm4"
Jul 14 11:19:32.399460 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.399415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:19:32.401750 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.401728    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 11:19:32.412414 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.412391    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0ffcf77-e340-4c18-96d2-c069f045d964-original-pull-secret\") pod \"global-pull-secret-syncer-cbj6p\" (UID: \"e0ffcf77-e340-4c18-96d2-c069f045d964\") " pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:19:32.499936 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.499895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:19:32.502172 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.502145    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 11:19:32.513493 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.513467    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab6dd4be-d083-4d37-b6b8-16abfcb09760-metrics-certs\") pod \"network-metrics-daemon-89z9c\" (UID: \"ab6dd4be-d083-4d37-b6b8-16abfcb09760\") " pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:19:32.514401 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.514378    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"]
Jul 14 11:19:32.518122 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.518101    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.522458 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.522435    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 14 11:19:32.523199 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.523182    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 14 11:19:32.523450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.523434    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 11:19:32.523519 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.523449    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gx2c2\""
Jul 14 11:19:32.523519 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.523492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 11:19:32.523829 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.523809    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 11:19:32.533624 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.533599    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"]
Jul 14 11:19:32.595837 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.595806    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-swcvq"]
Jul 14 11:19:32.597993 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.597975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.600924 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.600878    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.601060 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.600943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.601060 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.601023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2a5099-5319-4720-85db-4c54686b4293-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.601167 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.601092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:19:32.601167 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.601136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmhr6\" (UniqueName: \"kubernetes.io/projected/cc2a5099-5319-4720-85db-4c54686b4293-kube-api-access-fmhr6\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.603152 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.601507    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 11:19:32.603152 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.602327    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 11:19:32.603152 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.602382    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5244g\""
Jul 14 11:19:32.603152 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.602806    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 11:19:32.608717 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.608683    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 11:19:32.615662 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.615640    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 11:19:32.626719 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.626679    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpj89\" (UniqueName: \"kubernetes.io/projected/fed8361b-4c48-4a30-8c36-8349ef0667f2-kube-api-access-vpj89\") pod \"network-check-target-zp7vf\" (UID: \"fed8361b-4c48-4a30-8c36-8349ef0667f2\") " pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:19:32.677066 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.676975    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bm9fn\""
Jul 14 11:19:32.683681 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.683658    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q2rq4\""
Jul 14 11:19:32.685649 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.685619    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-89z9c"
Jul 14 11:19:32.687365 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.687339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cbj6p"
Jul 14 11:19:32.692122 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.692101    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:19:32.702739 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.702593    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2a5099-5319-4720-85db-4c54686b4293-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.703609 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.703413    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-root\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.703609 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.703553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-tls\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.704121 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704058    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-textfile\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.704591 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc2a5099-5319-4720-85db-4c54686b4293-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2jhg\" (UniqueName: \"kubernetes.io/projected/fb1dd50f-b737-4e06-b6d3-325405b290a8-kube-api-access-t2jhg\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1dd50f-b737-4e06-b6d3-325405b290a8-metrics-client-ca\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704797    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmhr6\" (UniqueName: \"kubernetes.io/projected/cc2a5099-5319-4720-85db-4c54686b4293-kube-api-access-fmhr6\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704826    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704887    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-wtmp\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-sys\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.704992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:32.705412    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 14 11:19:32.706073 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:32.705470    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-tls podName:cc2a5099-5319-4720-85db-4c54686b4293 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:33.205450927 +0000 UTC m=+67.006006662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-tjpkp" (UID: "cc2a5099-5319-4720-85db-4c54686b4293") : secret "openshift-state-metrics-tls" not found
Jul 14 11:19:32.709386 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.709341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.714961 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.714930    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmhr6\" (UniqueName: \"kubernetes.io/projected/cc2a5099-5319-4720-85db-4c54686b4293-kube-api-access-fmhr6\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:32.806333 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1dd50f-b737-4e06-b6d3-325405b290a8-metrics-client-ca\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806483 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-wtmp\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806483 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-sys\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806483 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806394    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806483 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806433    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806483 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806465    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-root\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806778 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-tls\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806778 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-textfile\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.806778 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2jhg\" (UniqueName: \"kubernetes.io/projected/fb1dd50f-b737-4e06-b6d3-325405b290a8-kube-api-access-t2jhg\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.807403 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.806992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-root\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.808292 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.807705    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-wtmp\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.808292 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.807708    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-textfile\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.808292 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.807759    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb1dd50f-b737-4e06-b6d3-325405b290a8-sys\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.808292 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.808242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb1dd50f-b737-4e06-b6d3-325405b290a8-metrics-client-ca\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.808536 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.808298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-accelerators-collector-config\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.809925 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.809864    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.811227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.811183    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fb1dd50f-b737-4e06-b6d3-325405b290a8-node-exporter-tls\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.820642 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.820583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2jhg\" (UniqueName: \"kubernetes.io/projected/fb1dd50f-b737-4e06-b6d3-325405b290a8-kube-api-access-t2jhg\") pod \"node-exporter-swcvq\" (UID: \"fb1dd50f-b737-4e06-b6d3-325405b290a8\") " pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.851080 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.851045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cbj6p"]
Jul 14 11:19:32.854963 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:32.854925    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ffcf77_e340_4c18_96d2_c069f045d964.slice/crio-8504c6864d052214bb4ecba204907626c8df4292c6a31fabc387689c6efc8db0 WatchSource:0}: Error finding container 8504c6864d052214bb4ecba204907626c8df4292c6a31fabc387689c6efc8db0: Status 404 returned error can't find the container with id 8504c6864d052214bb4ecba204907626c8df4292c6a31fabc387689c6efc8db0
Jul 14 11:19:32.875105 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.875066    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zp7vf"]
Jul 14 11:19:32.879313 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:32.879167    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed8361b_4c48_4a30_8c36_8349ef0667f2.slice/crio-3141ae0690c563e9b3005265b7eedccfdea957ff65ba953eb7aad40ba0e635fe WatchSource:0}: Error finding container 3141ae0690c563e9b3005265b7eedccfdea957ff65ba953eb7aad40ba0e635fe: Status 404 returned error can't find the container with id 3141ae0690c563e9b3005265b7eedccfdea957ff65ba953eb7aad40ba0e635fe
Jul 14 11:19:32.915537 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:32.915508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-swcvq"
Jul 14 11:19:32.927766 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:32.927686    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1dd50f_b737_4e06_b6d3_325405b290a8.slice/crio-d5cd628d1cf023ded6abeda634cd06fbd997ee85dfad983e6dabbec01b6c08a7 WatchSource:0}: Error finding container d5cd628d1cf023ded6abeda634cd06fbd997ee85dfad983e6dabbec01b6c08a7: Status 404 returned error can't find the container with id d5cd628d1cf023ded6abeda634cd06fbd997ee85dfad983e6dabbec01b6c08a7
Jul 14 11:19:33.051244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.051215    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z5f9m"
Jul 14 11:19:33.079206 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.079169    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-89z9c"]
Jul 14 11:19:33.082604 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:33.082576    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6dd4be_d083_4d37_b6b8_16abfcb09760.slice/crio-7c29a8193722439001b98af29cc3568e4c8cce6dceaa123784e329ee882a132c WatchSource:0}: Error finding container 7c29a8193722439001b98af29cc3568e4c8cce6dceaa123784e329ee882a132c: Status 404 returned error can't find the container with id 7c29a8193722439001b98af29cc3568e4c8cce6dceaa123784e329ee882a132c
Jul 14 11:19:33.088195 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.088164    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbj6p" event={"ID":"e0ffcf77-e340-4c18-96d2-c069f045d964","Type":"ContainerStarted","Data":"8504c6864d052214bb4ecba204907626c8df4292c6a31fabc387689c6efc8db0"}
Jul 14 11:19:33.089357 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.089336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-89z9c" event={"ID":"ab6dd4be-d083-4d37-b6b8-16abfcb09760","Type":"ContainerStarted","Data":"7c29a8193722439001b98af29cc3568e4c8cce6dceaa123784e329ee882a132c"}
Jul 14 11:19:33.090630 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.090605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swcvq" event={"ID":"fb1dd50f-b737-4e06-b6d3-325405b290a8","Type":"ContainerStarted","Data":"d5cd628d1cf023ded6abeda634cd06fbd997ee85dfad983e6dabbec01b6c08a7"}
Jul 14 11:19:33.091636 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.091616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zp7vf" event={"ID":"fed8361b-4c48-4a30-8c36-8349ef0667f2","Type":"ContainerStarted","Data":"3141ae0690c563e9b3005265b7eedccfdea957ff65ba953eb7aad40ba0e635fe"}
Jul 14 11:19:33.210173 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.210095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:33.214295 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.214263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc2a5099-5319-4720-85db-4c54686b4293-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-tjpkp\" (UID: \"cc2a5099-5319-4720-85db-4c54686b4293\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:33.430021 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.429989    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"
Jul 14 11:19:33.635978 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.635891    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp"]
Jul 14 11:19:33.646231 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.646204    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 11:19:33.651298 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.651273    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.655093 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.654393    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 14 11:19:33.655093 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.654795    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 14 11:19:33.655093 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.654885    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 14 11:19:33.655093 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.654978    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 14 11:19:33.655093 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.654995    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 14 11:19:33.655389 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.655243    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s4x5m\""
Jul 14 11:19:33.655449 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.655415    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 14 11:19:33.655449 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.655418    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 14 11:19:33.655606 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.655581    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 14 11:19:33.657482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.657152    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 14 11:19:33.680447 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.671588    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 11:19:33.714850 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.714812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.714870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.714906    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.714931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-web-config\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.714957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.714981    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.715014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkvf9\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-kube-api-access-xkvf9\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.715046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.715103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-volume\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.715126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.715151    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.715176    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-out\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.715275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.715200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.815994 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.815963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816135 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-web-config\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816135 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816135 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816083    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816135 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816117    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkvf9\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-kube-api-access-xkvf9\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816146    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-volume\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-out\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.816411 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.816386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.820021 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.818604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.820021 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.818900    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.820021 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.819279    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.820021 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:33.819883    2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 14 11:19:33.820021 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:33.819959    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls podName:33bffc21-7fda-4b7b-b2cf-3b1594ec5174 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:34.319938155 +0000 UTC m=+68.120493888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174") : secret "alertmanager-main-tls" not found
Jul 14 11:19:33.821860 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.821814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.823526 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.823434    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.824772 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.824729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-volume\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.825146 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.825108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.825425 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.825382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.825977 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.825939    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-out\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.826562 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.826541    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-web-config\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.827217 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.827178    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:33.849240 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:33.849174    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkvf9\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-kube-api-access-xkvf9\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:34.100444 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:34.100360    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp" event={"ID":"cc2a5099-5319-4720-85db-4c54686b4293","Type":"ContainerStarted","Data":"8db42bfa2640b680f69e087168f266fb5318bad82754937a0bfac434c593f09f"}
Jul 14 11:19:34.100444 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:34.100431    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp" event={"ID":"cc2a5099-5319-4720-85db-4c54686b4293","Type":"ContainerStarted","Data":"0d5e956d93e5fe777ce8aca5212ff2b3f0d6289380d54d32863fef84096c27c6"}
Jul 14 11:19:34.100660 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:34.100460    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp" event={"ID":"cc2a5099-5319-4720-85db-4c54686b4293","Type":"ContainerStarted","Data":"547ddb8e0d79138c29f709158d381034dc853ef2b57aa9248ed40b575e252c39"}
Jul 14 11:19:34.322409 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:34.321812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:34.322409 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:34.321983    2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 14 11:19:34.322409 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:34.322049    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls podName:33bffc21-7fda-4b7b-b2cf-3b1594ec5174 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:35.322029322 +0000 UTC m=+69.122585054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174") : secret "alertmanager-main-tls" not found
Jul 14 11:19:35.332225 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.332101    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:35.335253 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.335221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:35.471188 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.471148    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:19:35.553524 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.552335    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-545bdb9697-c68hp"]
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.575991    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-545bdb9697-c68hp"]
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.576149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.580048    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.580360    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c496v24h26pqk\""
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.580533    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.580745    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.580939    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vtl4j\""
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.581143    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 14 11:19:35.588768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.581810    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635224    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k82ts\" (UniqueName: \"kubernetes.io/projected/c97234cd-3afe-47fc-85cd-be69ea9b3305-kube-api-access-k82ts\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-tls\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635314    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-grpc-tls\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635435    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c97234cd-3afe-47fc-85cd-be69ea9b3305-metrics-client-ca\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.635625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.635485    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736643 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-grpc-tls\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736643 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736645    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736898 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736898 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736736    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c97234cd-3afe-47fc-85cd-be69ea9b3305-metrics-client-ca\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736898 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736898 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k82ts\" (UniqueName: \"kubernetes.io/projected/c97234cd-3afe-47fc-85cd-be69ea9b3305-kube-api-access-k82ts\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736898 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736850    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-tls\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.736898 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.736876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.738378 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.738276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c97234cd-3afe-47fc-85cd-be69ea9b3305-metrics-client-ca\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.740797 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.740456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.740797 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.740685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-tls\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.741239 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.741198    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.741582 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.741546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.742864 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.742575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-grpc-tls\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.743666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.743641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c97234cd-3afe-47fc-85cd-be69ea9b3305-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.755849 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.755816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k82ts\" (UniqueName: \"kubernetes.io/projected/c97234cd-3afe-47fc-85cd-be69ea9b3305-kube-api-access-k82ts\") pod \"thanos-querier-545bdb9697-c68hp\" (UID: \"c97234cd-3afe-47fc-85cd-be69ea9b3305\") " pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:35.903064 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:35.902975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:36.108550 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:36.108504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swcvq" event={"ID":"fb1dd50f-b737-4e06-b6d3-325405b290a8","Type":"ContainerStarted","Data":"90e79dc4083ad6363c2ff0fc7d6565af4dc49d68f919d644094f2187ddcfa7ac"}
Jul 14 11:19:36.110177 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:36.110138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-89z9c" event={"ID":"ab6dd4be-d083-4d37-b6b8-16abfcb09760","Type":"ContainerStarted","Data":"32e20bd79d0b032148e8c6f88914ecdb9ce5655c38c153fae09522b9d7ad5230"}
Jul 14 11:19:37.115058 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.114976    2564 generic.go:356] "Generic (PLEG): container finished" podID="fb1dd50f-b737-4e06-b6d3-325405b290a8" containerID="90e79dc4083ad6363c2ff0fc7d6565af4dc49d68f919d644094f2187ddcfa7ac" exitCode=0
Jul 14 11:19:37.115058 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.115052    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swcvq" event={"ID":"fb1dd50f-b737-4e06-b6d3-325405b290a8","Type":"ContainerDied","Data":"90e79dc4083ad6363c2ff0fc7d6565af4dc49d68f919d644094f2187ddcfa7ac"}
Jul 14 11:19:37.296816 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.296780    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"]
Jul 14 11:19:37.319223 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.319176    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"]
Jul 14 11:19:37.319386 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.319327    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:37.322445 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.322416    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 14 11:19:37.322727 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.322684    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bfnfn\""
Jul 14 11:19:37.451190 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.451111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d87f2b-e0e6-4d75-a261-db1b140eac5e-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-m7hz6\" (UID: \"54d87f2b-e0e6-4d75-a261-db1b140eac5e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:37.451190 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.451162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsns8\" (UniqueName: \"kubernetes.io/projected/54d87f2b-e0e6-4d75-a261-db1b140eac5e-kube-api-access-lsns8\") pod \"monitoring-plugin-5f68df7f58-m7hz6\" (UID: \"54d87f2b-e0e6-4d75-a261-db1b140eac5e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:37.552271 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.552228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d87f2b-e0e6-4d75-a261-db1b140eac5e-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-m7hz6\" (UID: \"54d87f2b-e0e6-4d75-a261-db1b140eac5e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:37.552271 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.552279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsns8\" (UniqueName: \"kubernetes.io/projected/54d87f2b-e0e6-4d75-a261-db1b140eac5e-kube-api-access-lsns8\") pod \"monitoring-plugin-5f68df7f58-m7hz6\" (UID: \"54d87f2b-e0e6-4d75-a261-db1b140eac5e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:37.552528 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:37.552431    2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 14 11:19:37.552528 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:19:37.552517    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54d87f2b-e0e6-4d75-a261-db1b140eac5e-monitoring-plugin-cert podName:54d87f2b-e0e6-4d75-a261-db1b140eac5e nodeName:}" failed. No retries permitted until 2026-07-14 11:19:38.052491832 +0000 UTC m=+71.853047568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/54d87f2b-e0e6-4d75-a261-db1b140eac5e-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-m7hz6" (UID: "54d87f2b-e0e6-4d75-a261-db1b140eac5e") : secret "monitoring-plugin-cert" not found
Jul 14 11:19:37.562079 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:37.562012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsns8\" (UniqueName: \"kubernetes.io/projected/54d87f2b-e0e6-4d75-a261-db1b140eac5e-kube-api-access-lsns8\") pod \"monitoring-plugin-5f68df7f58-m7hz6\" (UID: \"54d87f2b-e0e6-4d75-a261-db1b140eac5e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:38.057223 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.057184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d87f2b-e0e6-4d75-a261-db1b140eac5e-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-m7hz6\" (UID: \"54d87f2b-e0e6-4d75-a261-db1b140eac5e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:38.059797 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.059766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/54d87f2b-e0e6-4d75-a261-db1b140eac5e-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-m7hz6\" (UID: \"54d87f2b-e0e6-4d75-a261-db1b140eac5e\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:38.231505 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.231458    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:38.661867 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.661773    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-545bdb9697-c68hp"]
Jul 14 11:19:38.669113 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:38.669089    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97234cd_3afe_47fc_85cd_be69ea9b3305.slice/crio-ed624153685c32892c7de024b81d0b42a86107769c255fe85906606cab4e7d6d WatchSource:0}: Error finding container ed624153685c32892c7de024b81d0b42a86107769c255fe85906606cab4e7d6d: Status 404 returned error can't find the container with id ed624153685c32892c7de024b81d0b42a86107769c255fe85906606cab4e7d6d
Jul 14 11:19:38.686753 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.686725    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"]
Jul 14 11:19:38.725217 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.725020    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 11:19:38.938281 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.938209    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:19:38.941993 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.941975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:38.949110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.948756    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 11:19:38.949110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.948810    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 11:19:38.949110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.949066    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-19sfo4q9nmuql\""
Jul 14 11:19:38.949110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.949097    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 11:19:38.949440 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.949416    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 11:19:38.949578 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.949456    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 11:19:38.949578 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.949483    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 11:19:38.950306 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.950292    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 11:19:38.950530 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.950503    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 11:19:38.950530 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.950523    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 11:19:38.950530 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.950508    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 11:19:38.950795 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.950779    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 11:19:38.951136 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.951119    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 11:19:38.951243 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.951125    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 11:19:38.951389 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.951372    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qtsxm\""
Jul 14 11:19:38.956571 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.956546    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 11:19:38.957831 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.957815    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 11:19:38.977402 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:38.977369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:19:39.064565 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.064764 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.064764 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.064764 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.064764 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.064764 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lt5h\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-kube-api-access-2lt5h\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064853    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065002 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.064989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.065019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.065043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.065084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.065114    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.065133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.065166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.065283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.065188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.129360 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.129324    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swcvq" event={"ID":"fb1dd50f-b737-4e06-b6d3-325405b290a8","Type":"ContainerStarted","Data":"ca675245e6b995237db2e60511c3bf949817a572277d632a235aafb19300997b"}
Jul 14 11:19:39.129553 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.129390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-swcvq" event={"ID":"fb1dd50f-b737-4e06-b6d3-325405b290a8","Type":"ContainerStarted","Data":"dec33a79ab30f70c57acdd4b9819e35568455648d726b99a8e068d6d20df2d55"}
Jul 14 11:19:39.131109 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.131082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zp7vf" event={"ID":"fed8361b-4c48-4a30-8c36-8349ef0667f2","Type":"ContainerStarted","Data":"cc76ec61aa1d8a8f4d4003a2ff0c25d942f235a351a1eef952b1b362a9069966"}
Jul 14 11:19:39.131244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.131148    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:19:39.132659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.132629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6" event={"ID":"54d87f2b-e0e6-4d75-a261-db1b140eac5e","Type":"ContainerStarted","Data":"1ec1c8d18c931bf1bdebe17a11ee15809a38d6a924567e77430ce886d6b1543b"}
Jul 14 11:19:39.134279 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.134254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cbj6p" event={"ID":"e0ffcf77-e340-4c18-96d2-c069f045d964","Type":"ContainerStarted","Data":"8833b2200f07fae22cf34abfafed96b474b5ab85181a6266c0eaea5d91c10edb"}
Jul 14 11:19:39.136203 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.136173    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-89z9c" event={"ID":"ab6dd4be-d083-4d37-b6b8-16abfcb09760","Type":"ContainerStarted","Data":"e7e2993afe956270082a259011401c6f4999a41fb6b741a9a2a981f9d92a7844"}
Jul 14 11:19:39.138415 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.138390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp" event={"ID":"cc2a5099-5319-4720-85db-4c54686b4293","Type":"ContainerStarted","Data":"f66c85383f01ecec95bd85de486bdb533a70b6df1e1f14334ae5aced10265c05"}
Jul 14 11:19:39.139746 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.139724    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerStarted","Data":"9c7feaaddd937c4903476fe5b2866757eca5fd7b9511dc79c8f31f36e233df7c"}
Jul 14 11:19:39.141015 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.140993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" event={"ID":"c97234cd-3afe-47fc-85cd-be69ea9b3305","Type":"ContainerStarted","Data":"ed624153685c32892c7de024b81d0b42a86107769c255fe85906606cab4e7d6d"}
Jul 14 11:19:39.155574 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.155521    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-swcvq" podStartSLOduration=4.873807558 podStartE2EDuration="7.155488751s" totalImagesPullingTime="2.159347318s" totalInitContainerRuntime="122.333875ms" isStatefulPod=true podCreationTimestamp="2026-07-14 11:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:39.154640183 +0000 UTC m=+72.955195941" watchObservedRunningTime="2026-07-14 11:19:39.155488751 +0000 UTC m=+72.956044470"
Jul 14 11:19:39.166601 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.166476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.166601 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.166518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.166601 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.166550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167010 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.166985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167086 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167086 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167062    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167290 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167181    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167290 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lt5h\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-kube-api-access-2lt5h\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167290 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167245    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167290 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167340    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167366    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167402    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167803 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167803 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167803 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.167803 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.168010 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.168010 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.167901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.168457 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.168433    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.168549 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.168437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.170388 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.170351    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.170627 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.170445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.171469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.170803    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.171469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.171170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.171469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.171283    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cbj6p" podStartSLOduration=66.564872334 podStartE2EDuration="1m12.171267162s" totalImagesPullingTime="5.606394828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:39.171046392 +0000 UTC m=+72.971602133" watchObservedRunningTime="2026-07-14 11:19:39.171267162 +0000 UTC m=+72.971822904"
Jul 14 11:19:39.171469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.171338    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.171746 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.171561    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.172292 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.172248    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.173856 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.173823    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-web-config\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.174357 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.174330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.174449 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.174395    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.174926 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.174905    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.175329 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.175309    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.176840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.176796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config-out\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.178201 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.178179    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lt5h\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-kube-api-access-2lt5h\") pod \"prometheus-k8s-0\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.192014 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.191918    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-89z9c" podStartSLOduration=70.184783332 podStartE2EDuration="1m12.191898639s" totalImagesPullingTime="2.007115307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:39.189565057 +0000 UTC m=+72.990120798" watchObservedRunningTime="2026-07-14 11:19:39.191898639 +0000 UTC m=+72.992454381"
Jul 14 11:19:39.210450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.210386    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zp7vf" podStartSLOduration=66.640464285 podStartE2EDuration="1m12.210364927s" totalImagesPullingTime="5.569900642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:39.20839667 +0000 UTC m=+73.008952437" watchObservedRunningTime="2026-07-14 11:19:39.210364927 +0000 UTC m=+73.010920669"
Jul 14 11:19:39.233971 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.233915    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-tjpkp" podStartSLOduration=2.738509612 podStartE2EDuration="7.233894629s" totalImagesPullingTime="4.495385017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:39.232882383 +0000 UTC m=+73.033438145" watchObservedRunningTime="2026-07-14 11:19:39.233894629 +0000 UTC m=+73.034450368"
Jul 14 11:19:39.252352 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.252317    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:19:39.433128 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:39.432966    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:19:39.436604 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:19:39.436569    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f06bb59_c3f6_4e7f_984a_e64932b9cf59.slice/crio-b976d6270ee464b6c58a351b9bee777d6d391730912d0060af26824e66922be0 WatchSource:0}: Error finding container b976d6270ee464b6c58a351b9bee777d6d391730912d0060af26824e66922be0: Status 404 returned error can't find the container with id b976d6270ee464b6c58a351b9bee777d6d391730912d0060af26824e66922be0
Jul 14 11:19:40.146680 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:40.146502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerStarted","Data":"b976d6270ee464b6c58a351b9bee777d6d391730912d0060af26824e66922be0"}
Jul 14 11:19:41.151824 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.151787    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6" event={"ID":"54d87f2b-e0e6-4d75-a261-db1b140eac5e","Type":"ContainerStarted","Data":"5112cac2117424bc1f85bc41412637cc8f89b297013a44091fa74d74c9366cfd"}
Jul 14 11:19:41.152272 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.151995    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:41.153647 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.153619    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerID="2bd50b0fa0c3db5077784826d376fd592a551dbea31cdb438c66451016bb37a4" exitCode=0
Jul 14 11:19:41.153797 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.153716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"2bd50b0fa0c3db5077784826d376fd592a551dbea31cdb438c66451016bb37a4"}
Jul 14 11:19:41.155380 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.155355    2564 generic.go:356] "Generic (PLEG): container finished" podID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerID="25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f" exitCode=0
Jul 14 11:19:41.155497 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.155429    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f"}
Jul 14 11:19:41.157668 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.157630    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6"
Jul 14 11:19:41.157864 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.157845    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" event={"ID":"c97234cd-3afe-47fc-85cd-be69ea9b3305","Type":"ContainerStarted","Data":"bf372e5bc5d7d50f071578b36beb611e7d4a361500034e72375f289e0a1c2c86"}
Jul 14 11:19:41.157943 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.157906    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" event={"ID":"c97234cd-3afe-47fc-85cd-be69ea9b3305","Type":"ContainerStarted","Data":"3cdbf877527320dd20cf4f98cc62b468774c004c6ef8c255629506c4ab681ca7"}
Jul 14 11:19:41.157997 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.157954    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" event={"ID":"c97234cd-3afe-47fc-85cd-be69ea9b3305","Type":"ContainerStarted","Data":"9cf9958d0e900e96e7392a497dca6e6230775d9b956c51e50e5c383ecd89fe21"}
Jul 14 11:19:41.169516 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:41.169471    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-m7hz6" podStartSLOduration=2.719054232 podStartE2EDuration="4.169457027s" totalImagesPullingTime="1.450402795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:41.16871942 +0000 UTC m=+74.969275158" watchObservedRunningTime="2026-07-14 11:19:41.169457027 +0000 UTC m=+74.970012816"
Jul 14 11:19:42.167162 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:42.167043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" event={"ID":"c97234cd-3afe-47fc-85cd-be69ea9b3305","Type":"ContainerStarted","Data":"96b857926cd6b084a0c099426cc7f79dacc2ec0fa619f8d825ed1d53d2747517"}
Jul 14 11:19:42.167673 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:42.167170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" event={"ID":"c97234cd-3afe-47fc-85cd-be69ea9b3305","Type":"ContainerStarted","Data":"88440f5bf334326bc62be95a484ebb4c907d4f77982caf06524043add03b3f1f"}
Jul 14 11:19:42.167673 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:42.167215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" event={"ID":"c97234cd-3afe-47fc-85cd-be69ea9b3305","Type":"ContainerStarted","Data":"d77ca62573af815aad3b6fb301deed4605eb5a3be6a1c8bbe53b35ddb13d5d97"}
Jul 14 11:19:42.193532 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:42.192820    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp" podStartSLOduration=4.314925695 podStartE2EDuration="7.192797077s" totalImagesPullingTime="2.877871382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:42.19116015 +0000 UTC m=+75.991715890" watchObservedRunningTime="2026-07-14 11:19:42.192797077 +0000 UTC m=+75.993352818"
Jul 14 11:19:43.175081 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:43.175046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerStarted","Data":"2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0"}
Jul 14 11:19:43.175512 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:43.175133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerStarted","Data":"44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6"}
Jul 14 11:19:43.175512 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:43.175193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerStarted","Data":"92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698"}
Jul 14 11:19:43.175512 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:43.175262    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:44.054895 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:44.054864    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b66646578-mlbpl"
Jul 14 11:19:44.184049 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:44.184016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerStarted","Data":"cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71"}
Jul 14 11:19:44.184550 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:44.184092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerStarted","Data":"e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8"}
Jul 14 11:19:44.184550 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:44.184133    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerStarted","Data":"80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b"}
Jul 14 11:19:44.219667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:44.219607    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=8.02463569 podStartE2EDuration="11.219587875s" totalImagesPullingTime="3.175656597s" totalInitContainerRuntime="19.295588ms" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:44.219036151 +0000 UTC m=+78.019591890" watchObservedRunningTime="2026-07-14 11:19:44.219587875 +0000 UTC m=+78.020143615"
Jul 14 11:19:45.191644 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:45.191603    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerStarted","Data":"14b8c1025bd5498597b1a424d6c44980921bf7fe86cd79ffd4d2fdf8a512184b"}
Jul 14 11:19:45.192023 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:45.191676    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerStarted","Data":"9fcb9bce19145e8feb8dfb6fc06c260a1fa1f69e21893d91f4b2ed564041097a"}
Jul 14 11:19:45.192023 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:45.191721    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerStarted","Data":"ed6b98ef03c8a086444a95001b55996bb6398fddedbdce6664d74e9b7d508e15"}
Jul 14 11:19:45.192023 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:45.191750    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerStarted","Data":"824b6dcb8e676639b02af5329dd51c1b5ad967fd9474dae5f0d146ea786567c6"}
Jul 14 11:19:45.192023 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:45.191773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerStarted","Data":"16d636144835908ae240a3c5bb8b0a7c9139e4a6833c032c6a6d37eeb49d6728"}
Jul 14 11:19:45.192023 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:45.191795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerStarted","Data":"98a72595e8383326dce034c89ff8c4f486134df947bde9218984d1794f3213bc"}
Jul 14 11:19:45.224868 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:45.224817    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.159286131 podStartE2EDuration="7.224798372s" totalImagesPullingTime="4.043125029s" totalInitContainerRuntime="22.387212ms" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:45.223058118 +0000 UTC m=+79.023613916" watchObservedRunningTime="2026-07-14 11:19:45.224798372 +0000 UTC m=+79.025354112"
Jul 14 11:19:49.190129 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:49.190013    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-545bdb9697-c68hp"
Jul 14 11:19:49.253314 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:19:49.253279    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:05.012301 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:05.012270    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_33bffc21-7fda-4b7b-b2cf-3b1594ec5174/init-config-reloader/0.log"
Jul 14 11:20:05.211779 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:05.211753    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_33bffc21-7fda-4b7b-b2cf-3b1594ec5174/alertmanager/0.log"
Jul 14 11:20:05.420200 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:05.420176    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_33bffc21-7fda-4b7b-b2cf-3b1594ec5174/config-reloader/0.log"
Jul 14 11:20:05.616127 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:05.616096    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_33bffc21-7fda-4b7b-b2cf-3b1594ec5174/kube-rbac-proxy-web/0.log"
Jul 14 11:20:05.812505 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:05.812475    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_33bffc21-7fda-4b7b-b2cf-3b1594ec5174/kube-rbac-proxy/0.log"
Jul 14 11:20:06.012339 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:06.012306    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_33bffc21-7fda-4b7b-b2cf-3b1594ec5174/kube-rbac-proxy-metric/0.log"
Jul 14 11:20:06.211750 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:06.211648    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_33bffc21-7fda-4b7b-b2cf-3b1594ec5174/prom-label-proxy/0.log"
Jul 14 11:20:07.412096 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:07.412056    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-m7hz6_54d87f2b-e0e6-4d75-a261-db1b140eac5e/monitoring-plugin/0.log"
Jul 14 11:20:08.812665 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:08.812636    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swcvq_fb1dd50f-b737-4e06-b6d3-325405b290a8/init-textfile/0.log"
Jul 14 11:20:09.012717 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:09.012670    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swcvq_fb1dd50f-b737-4e06-b6d3-325405b290a8/node-exporter/0.log"
Jul 14 11:20:09.212062 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:09.211988    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swcvq_fb1dd50f-b737-4e06-b6d3-325405b290a8/kube-rbac-proxy/0.log"
Jul 14 11:20:09.415095 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:09.415067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tjpkp_cc2a5099-5319-4720-85db-4c54686b4293/kube-rbac-proxy-main/0.log"
Jul 14 11:20:09.614824 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:09.614794    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tjpkp_cc2a5099-5319-4720-85db-4c54686b4293/kube-rbac-proxy-self/0.log"
Jul 14 11:20:09.813435 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:09.813404    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tjpkp_cc2a5099-5319-4720-85db-4c54686b4293/openshift-state-metrics/0.log"
Jul 14 11:20:10.011251 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:10.011168    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f06bb59-c3f6-4e7f-984a-e64932b9cf59/init-config-reloader/0.log"
Jul 14 11:20:10.149779 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:10.149753    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zp7vf"
Jul 14 11:20:10.213579 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:10.213554    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f06bb59-c3f6-4e7f-984a-e64932b9cf59/prometheus/0.log"
Jul 14 11:20:10.412651 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:10.412619    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f06bb59-c3f6-4e7f-984a-e64932b9cf59/config-reloader/0.log"
Jul 14 11:20:10.611231 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:10.611203    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f06bb59-c3f6-4e7f-984a-e64932b9cf59/thanos-sidecar/0.log"
Jul 14 11:20:10.816325 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:10.816285    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f06bb59-c3f6-4e7f-984a-e64932b9cf59/kube-rbac-proxy-web/0.log"
Jul 14 11:20:11.016879 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:11.016845    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f06bb59-c3f6-4e7f-984a-e64932b9cf59/kube-rbac-proxy/0.log"
Jul 14 11:20:11.211802 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:11.211726    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4f06bb59-c3f6-4e7f-984a-e64932b9cf59/kube-rbac-proxy-thanos/0.log"
Jul 14 11:20:11.811073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:11.811041    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-7p5z4_67309b0e-d8d1-4b17-83f9-d7712eede2b0/prometheus-operator-admission-webhook/0.log"
Jul 14 11:20:12.012029 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:12.011999    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/thanos-query/0.log"
Jul 14 11:20:12.211193 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:12.211119    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy-web/0.log"
Jul 14 11:20:12.411662 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:12.411636    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy/0.log"
Jul 14 11:20:12.611943 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:12.611918    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/prom-label-proxy/0.log"
Jul 14 11:20:12.811524 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:12.811499    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy-rules/0.log"
Jul 14 11:20:13.012456 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:13.012373    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy-metrics/0.log"
Jul 14 11:20:14.611620 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:14.611586    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5wpn6_813f9693-f3e7-4c38-8ed1-7be66382f4ef/serve-healthcheck-canary/0.log"
Jul 14 11:20:39.253094 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:39.253055    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:39.272034 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:39.272011    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:39.420214 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:39.420186    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:52.860558 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:52.860520    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 11:20:52.861151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:52.860911    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="alertmanager" containerID="cri-o://92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698" gracePeriod=120
Jul 14 11:20:52.861151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:52.860961    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-metric" containerID="cri-o://80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b" gracePeriod=120
Jul 14 11:20:52.861151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:52.860981    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-web" containerID="cri-o://2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0" gracePeriod=120
Jul 14 11:20:52.861151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:52.860981    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="config-reloader" containerID="cri-o://44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6" gracePeriod=120
Jul 14 11:20:52.861151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:52.860999    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy" containerID="cri-o://cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71" gracePeriod=120
Jul 14 11:20:52.861151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:52.861047    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="prom-label-proxy" containerID="cri-o://e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8" gracePeriod=120
Jul 14 11:20:53.463109 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463074    2564 generic.go:356] "Generic (PLEG): container finished" podID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerID="e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8" exitCode=0
Jul 14 11:20:53.463109 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463101    2564 generic.go:356] "Generic (PLEG): container finished" podID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerID="80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b" exitCode=0
Jul 14 11:20:53.463109 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463107    2564 generic.go:356] "Generic (PLEG): container finished" podID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerID="cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71" exitCode=0
Jul 14 11:20:53.463109 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463114    2564 generic.go:356] "Generic (PLEG): container finished" podID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerID="44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6" exitCode=0
Jul 14 11:20:53.463109 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463119    2564 generic.go:356] "Generic (PLEG): container finished" podID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerID="92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698" exitCode=0
Jul 14 11:20:53.463479 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463148    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8"}
Jul 14 11:20:53.463479 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b"}
Jul 14 11:20:53.463479 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71"}
Jul 14 11:20:53.463479 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6"}
Jul 14 11:20:53.463479 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:53.463329    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698"}
Jul 14 11:20:54.114481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.114456    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:20:54.210782 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.210678    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-out\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.210782 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.210730    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-tls-assets\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.210782 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.210771    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-cluster-tls-config\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211058 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.210805    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211058 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.210848    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-web-config\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211058 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.210960    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-trusted-ca-bundle\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211058 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211014    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-metrics-client-ca\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211065    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-web\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211106    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkvf9\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-kube-api-access-xkvf9\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211131    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211169    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-volume\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211197    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211226    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-main-db\") pod \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\" (UID: \"33bffc21-7fda-4b7b-b2cf-3b1594ec5174\") "
Jul 14 11:20:54.211562 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211539    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:54.211804 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.211779    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:20:54.212533 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.212491    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:54.214048 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.214017    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:54.214178 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.214098    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:20:54.214319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.214271    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-out" (OuterVolumeSpecName: "config-out") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:20:54.214758 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.214717    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:54.214922 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.214888    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-volume" (OuterVolumeSpecName: "config-volume") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:54.215000 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.214954    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:54.215610 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.215588    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-kube-api-access-xkvf9" (OuterVolumeSpecName: "kube-api-access-xkvf9") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "kube-api-access-xkvf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:20:54.215689 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.215648    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:54.218247 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.218220    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:54.224670 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.224647    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-web-config" (OuterVolumeSpecName: "web-config") pod "33bffc21-7fda-4b7b-b2cf-3b1594ec5174" (UID: "33bffc21-7fda-4b7b-b2cf-3b1594ec5174"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:54.312322 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312291    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-web-config\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312322 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312318    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312322 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312327    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-metrics-client-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312338    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312347    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkvf9\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-kube-api-access-xkvf9\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312357    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-main-tls\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312365    2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-volume\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312373    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312381    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-alertmanager-main-db\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312389    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-config-out\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312398    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-tls-assets\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312405    2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-cluster-tls-config\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.312539 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.312420    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/33bffc21-7fda-4b7b-b2cf-3b1594ec5174-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:54.469805 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.469715    2564 generic.go:356] "Generic (PLEG): container finished" podID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerID="2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0" exitCode=0
Jul 14 11:20:54.469805 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.469800    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0"}
Jul 14 11:20:54.469981 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.469821    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 11:20:54.469981 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.469873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"33bffc21-7fda-4b7b-b2cf-3b1594ec5174","Type":"ContainerDied","Data":"9c7feaaddd937c4903476fe5b2866757eca5fd7b9511dc79c8f31f36e233df7c"}
Jul 14 11:20:54.469981 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.469920    2564 scope.go:117] "RemoveContainer" containerID="e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8"
Jul 14 11:20:54.477853 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.477788    2564 scope.go:117] "RemoveContainer" containerID="80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b"
Jul 14 11:20:54.484571 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.484556    2564 scope.go:117] "RemoveContainer" containerID="cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71"
Jul 14 11:20:54.491374 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.491350    2564 scope.go:117] "RemoveContainer" containerID="2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0"
Jul 14 11:20:54.492824 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.492804    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 11:20:54.499178 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.499129    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 11:20:54.499178 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.499163    2564 scope.go:117] "RemoveContainer" containerID="44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6"
Jul 14 11:20:54.505793 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.505775    2564 scope.go:117] "RemoveContainer" containerID="92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698"
Jul 14 11:20:54.512570 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.512553    2564 scope.go:117] "RemoveContainer" containerID="25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f"
Jul 14 11:20:54.519384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.519366    2564 scope.go:117] "RemoveContainer" containerID="e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8"
Jul 14 11:20:54.519648 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:20:54.519629    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8\": container with ID starting with e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8 not found: ID does not exist" containerID="e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8"
Jul 14 11:20:54.519717 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.519656    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8"} err="failed to get container status \"e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8\": rpc error: code = NotFound desc = could not find container \"e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8\": container with ID starting with e89f7a2b39fe4638f933844df0db0d8b1fb800b76cea3fe3eb75453bb6cd44d8 not found: ID does not exist"
Jul 14 11:20:54.519717 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.519687    2564 scope.go:117] "RemoveContainer" containerID="80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b"
Jul 14 11:20:54.519922 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:20:54.519901    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b\": container with ID starting with 80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b not found: ID does not exist" containerID="80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b"
Jul 14 11:20:54.519959 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.519929    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b"} err="failed to get container status \"80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b\": rpc error: code = NotFound desc = could not find container \"80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b\": container with ID starting with 80d66a86109b68af25b883d01b6f3b0ce33237278eb15df9690fa4b35931c18b not found: ID does not exist"
Jul 14 11:20:54.519959 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.519947    2564 scope.go:117] "RemoveContainer" containerID="cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71"
Jul 14 11:20:54.520189 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:20:54.520172    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71\": container with ID starting with cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71 not found: ID does not exist" containerID="cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71"
Jul 14 11:20:54.520238 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.520193    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71"} err="failed to get container status \"cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71\": rpc error: code = NotFound desc = could not find container \"cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71\": container with ID starting with cab51055d71d97394806baebeea9af3b48ebda77049c8136422dda04dc459c71 not found: ID does not exist"
Jul 14 11:20:54.520238 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.520206    2564 scope.go:117] "RemoveContainer" containerID="2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0"
Jul 14 11:20:54.520432 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:20:54.520417    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0\": container with ID starting with 2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0 not found: ID does not exist" containerID="2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0"
Jul 14 11:20:54.520480 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.520436    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0"} err="failed to get container status \"2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0\": rpc error: code = NotFound desc = could not find container \"2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0\": container with ID starting with 2a414af4447edf3f7d5ca794074a118ffaa951c3c8c12b72ae639f213ac963e0 not found: ID does not exist"
Jul 14 11:20:54.520480 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.520449    2564 scope.go:117] "RemoveContainer" containerID="44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6"
Jul 14 11:20:54.520760 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:20:54.520742    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6\": container with ID starting with 44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6 not found: ID does not exist" containerID="44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6"
Jul 14 11:20:54.520822 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.520763    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6"} err="failed to get container status \"44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6\": rpc error: code = NotFound desc = could not find container \"44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6\": container with ID starting with 44c8172d38a8cf8cbb6a275e103568d06e06231b39e576e9a137df7bcb218ee6 not found: ID does not exist"
Jul 14 11:20:54.520822 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.520776    2564 scope.go:117] "RemoveContainer" containerID="92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698"
Jul 14 11:20:54.521001 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:20:54.520986    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698\": container with ID starting with 92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698 not found: ID does not exist" containerID="92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698"
Jul 14 11:20:54.521044 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.521005    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698"} err="failed to get container status \"92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698\": rpc error: code = NotFound desc = could not find container \"92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698\": container with ID starting with 92b0d569ab325f4a42e8909fffd51fc00c8507cd63477db86b540f06e122b698 not found: ID does not exist"
Jul 14 11:20:54.521044 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.521019    2564 scope.go:117] "RemoveContainer" containerID="25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f"
Jul 14 11:20:54.521228 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:20:54.521214    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f\": container with ID starting with 25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f not found: ID does not exist" containerID="25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f"
Jul 14 11:20:54.521269 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.521231    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f"} err="failed to get container status \"25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f\": rpc error: code = NotFound desc = could not find container \"25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f\": container with ID starting with 25e42cfa33ad9fee17e16859767d0d882b7b379f7c8bd1139c9dd1210f41b84f not found: ID does not exist"
Jul 14 11:20:54.768366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:54.768335    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" path="/var/lib/kubelet/pods/33bffc21-7fda-4b7b-b2cf-3b1594ec5174/volumes"
Jul 14 11:20:57.254974 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.254935    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:20:57.255466 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.255308    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="prometheus" containerID="cri-o://98a72595e8383326dce034c89ff8c4f486134df947bde9218984d1794f3213bc" gracePeriod=600
Jul 14 11:20:57.255466 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.255330    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy" containerID="cri-o://9fcb9bce19145e8feb8dfb6fc06c260a1fa1f69e21893d91f4b2ed564041097a" gracePeriod=600
Jul 14 11:20:57.255466 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.255365    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-thanos" containerID="cri-o://14b8c1025bd5498597b1a424d6c44980921bf7fe86cd79ffd4d2fdf8a512184b" gracePeriod=600
Jul 14 11:20:57.255466 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.255341    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="thanos-sidecar" containerID="cri-o://824b6dcb8e676639b02af5329dd51c1b5ad967fd9474dae5f0d146ea786567c6" gracePeriod=600
Jul 14 11:20:57.255663 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.255367    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="config-reloader" containerID="cri-o://16d636144835908ae240a3c5bb8b0a7c9139e4a6833c032c6a6d37eeb49d6728" gracePeriod=600
Jul 14 11:20:57.255663 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.255341    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-web" containerID="cri-o://ed6b98ef03c8a086444a95001b55996bb6398fddedbdce6664d74e9b7d508e15" gracePeriod=600
Jul 14 11:20:57.486816 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486783    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerID="14b8c1025bd5498597b1a424d6c44980921bf7fe86cd79ffd4d2fdf8a512184b" exitCode=0
Jul 14 11:20:57.486816 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486809    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerID="9fcb9bce19145e8feb8dfb6fc06c260a1fa1f69e21893d91f4b2ed564041097a" exitCode=0
Jul 14 11:20:57.486816 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486815    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerID="ed6b98ef03c8a086444a95001b55996bb6398fddedbdce6664d74e9b7d508e15" exitCode=0
Jul 14 11:20:57.486816 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486821    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerID="824b6dcb8e676639b02af5329dd51c1b5ad967fd9474dae5f0d146ea786567c6" exitCode=0
Jul 14 11:20:57.486816 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486826    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerID="16d636144835908ae240a3c5bb8b0a7c9139e4a6833c032c6a6d37eeb49d6728" exitCode=0
Jul 14 11:20:57.487081 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486831    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerID="98a72595e8383326dce034c89ff8c4f486134df947bde9218984d1794f3213bc" exitCode=0
Jul 14 11:20:57.487081 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486857    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"14b8c1025bd5498597b1a424d6c44980921bf7fe86cd79ffd4d2fdf8a512184b"}
Jul 14 11:20:57.487081 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.486970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"9fcb9bce19145e8feb8dfb6fc06c260a1fa1f69e21893d91f4b2ed564041097a"}
Jul 14 11:20:57.487081 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.487013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"ed6b98ef03c8a086444a95001b55996bb6398fddedbdce6664d74e9b7d508e15"}
Jul 14 11:20:57.487081 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.487051    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"824b6dcb8e676639b02af5329dd51c1b5ad967fd9474dae5f0d146ea786567c6"}
Jul 14 11:20:57.487244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.487089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"16d636144835908ae240a3c5bb8b0a7c9139e4a6833c032c6a6d37eeb49d6728"}
Jul 14 11:20:57.487244 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.487124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"98a72595e8383326dce034c89ff8c4f486134df947bde9218984d1794f3213bc"}
Jul 14 11:20:57.496342 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.496320    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:57.537189 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537155    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-metrics-client-ca\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537344 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537204    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config-out\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537344 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537232    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537344 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537261    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-grpc-tls\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537344 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537294    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-serving-certs-ca-bundle\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537344 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537320    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lt5h\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-kube-api-access-2lt5h\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537350    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-1\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537379    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-kubelet-serving-ca-bundle\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537404    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-tls-assets\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537433    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-tls\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537460    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-thanos-prometheus-http-client-file\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537487    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-web-config\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537527    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-trusted-ca-bundle\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537569    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-kube-rbac-proxy\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.537596 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537596    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.538046 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537615    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:57.538046 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537626    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-db\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.538046 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537809    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-2\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.538046 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537843    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-0\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.538046 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537878    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.538046 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.537908    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-metrics-client-certs\") pod \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\" (UID: \"4f06bb59-c3f6-4e7f-984a-e64932b9cf59\") "
Jul 14 11:20:57.538338 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.538154    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-metrics-client-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.538338 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.538164    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:57.538511 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.538483    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:57.540464 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.540228    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:57.540464 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.540420    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:57.540627 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.540478    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:57.540833 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.540803    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:20:57.541307 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.541006    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-kube-api-access-2lt5h" (OuterVolumeSpecName: "kube-api-access-2lt5h") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "kube-api-access-2lt5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:20:57.541307 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.541216    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:20:57.542293 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.542259    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.542434 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.542390    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.542942 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.542914    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.543351 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.543321    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.543439 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.543408    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.543836 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.543737    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:20:57.543920 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.543842    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.543996 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.543967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config-out" (OuterVolumeSpecName: "config-out") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:20:57.544206 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.544177    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config" (OuterVolumeSpecName: "config") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.544446 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.544425    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.553968 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.553948    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-web-config" (OuterVolumeSpecName: "web-config") pod "4f06bb59-c3f6-4e7f-984a-e64932b9cf59" (UID: "4f06bb59-c3f6-4e7f-984a-e64932b9cf59"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:20:57.638469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638437    2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-kube-rbac-proxy\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638467    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638478    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-db\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638487    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638495    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638504    2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638512    2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-metrics-client-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638521    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-config-out\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638530    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638539    2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-grpc-tls\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638547    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638557    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lt5h\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-kube-api-access-2lt5h\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638565    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638574    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638582    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-tls-assets\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638590    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.638619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638598    2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.639060 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638628    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-web-config\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:57.639060 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:57.638637    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f06bb59-c3f6-4e7f-984a-e64932b9cf59-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:20:58.493814 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.493772    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4f06bb59-c3f6-4e7f-984a-e64932b9cf59","Type":"ContainerDied","Data":"b976d6270ee464b6c58a351b9bee777d6d391730912d0060af26824e66922be0"}
Jul 14 11:20:58.493814 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.493819    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.494318 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.493873    2564 scope.go:117] "RemoveContainer" containerID="14b8c1025bd5498597b1a424d6c44980921bf7fe86cd79ffd4d2fdf8a512184b"
Jul 14 11:20:58.502455 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.502370    2564 scope.go:117] "RemoveContainer" containerID="9fcb9bce19145e8feb8dfb6fc06c260a1fa1f69e21893d91f4b2ed564041097a"
Jul 14 11:20:58.509935 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.509919    2564 scope.go:117] "RemoveContainer" containerID="ed6b98ef03c8a086444a95001b55996bb6398fddedbdce6664d74e9b7d508e15"
Jul 14 11:20:58.516572 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.516555    2564 scope.go:117] "RemoveContainer" containerID="824b6dcb8e676639b02af5329dd51c1b5ad967fd9474dae5f0d146ea786567c6"
Jul 14 11:20:58.521122 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.521069    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:20:58.523427 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.523412    2564 scope.go:117] "RemoveContainer" containerID="16d636144835908ae240a3c5bb8b0a7c9139e4a6833c032c6a6d37eeb49d6728"
Jul 14 11:20:58.527533 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.527514    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:20:58.532497 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.532475    2564 scope.go:117] "RemoveContainer" containerID="98a72595e8383326dce034c89ff8c4f486134df947bde9218984d1794f3213bc"
Jul 14 11:20:58.539450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.539436    2564 scope.go:117] "RemoveContainer" containerID="2bd50b0fa0c3db5077784826d376fd592a551dbea31cdb438c66451016bb37a4"
Jul 14 11:20:58.557430 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557407    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:20:58.557666 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557655    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557667    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557677    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-web"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557683    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-web"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557689    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557725    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557731    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-thanos"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557737    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-thanos"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557748    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="init-config-reloader"
Jul 14 11:20:58.557751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557753    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="init-config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557760    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-web"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557765    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-web"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557773    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="alertmanager"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557778    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="alertmanager"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557784    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="prom-label-proxy"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557790    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="prom-label-proxy"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557795    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557800    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557807    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-metric"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557813    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-metric"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557818    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="thanos-sidecar"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557823    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="thanos-sidecar"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557829    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="prometheus"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557834    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="prometheus"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557841    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557846    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557853    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="init-config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557859    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="init-config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557898    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557905    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557912    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-thanos"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557918    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-web"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557925    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="kube-rbac-proxy-metric"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557934    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="prom-label-proxy"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557942    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557950    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="kube-rbac-proxy-web"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557956    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="config-reloader"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557961    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bffc21-7fda-4b7b-b2cf-3b1594ec5174" containerName="alertmanager"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557967    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="thanos-sidecar"
Jul 14 11:20:58.558055 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.557971    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" containerName="prometheus"
Jul 14 11:20:58.561516 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.561498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.563948 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.563925    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 11:20:58.564142 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.564113    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 11:20:58.564442 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.564428    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 11:20:58.564605 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.564591    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 11:20:58.564739 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.564690    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 11:20:58.564855 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.564836    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qtsxm\""
Jul 14 11:20:58.564923 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.564911    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 11:20:58.564975 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.564928    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 11:20:58.565020 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.565000    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 11:20:58.565316 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.565295    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 11:20:58.565406 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.565319    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 11:20:58.565406 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.565327    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 11:20:58.565406 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.565299    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 11:20:58.565406 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.565322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 11:20:58.566915 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.566899    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-19sfo4q9nmuql\""
Jul 14 11:20:58.568796 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.568770    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 11:20:58.571842 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.571826    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 11:20:58.578780 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.578762    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:20:58.645629 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645587    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645629 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645627    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7039a24-5b18-4989-a851-617b918efdd9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645736    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645799    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645879    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.645918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.645983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646036    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646054    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-config\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7039a24-5b18-4989-a851-617b918efdd9-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646423 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc2r4\" (UniqueName: \"kubernetes.io/projected/d7039a24-5b18-4989-a851-617b918efdd9-kube-api-access-bc2r4\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646423 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646230    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.646423 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.646246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.746951 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.746916    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747067 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.746953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747067 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.746976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747067 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.746999    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747067 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747022    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747067 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747133    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-config\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7039a24-5b18-4989-a851-617b918efdd9-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc2r4\" (UniqueName: \"kubernetes.io/projected/d7039a24-5b18-4989-a851-617b918efdd9-kube-api-access-bc2r4\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747311 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747787 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747787 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747787 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747787 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7039a24-5b18-4989-a851-617b918efdd9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.747787 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.748032 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.747844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.748121 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.748090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.749863 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.749839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.750094 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.750073    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.750225 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.750178    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.750225 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.750128    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.751549 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.751459    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.751549 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.751485    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.751549 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.751538    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.751815 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.751592    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.751815 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.751613    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.752209 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.752157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.752209 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.752166    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7039a24-5b18-4989-a851-617b918efdd9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.752393 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.752368    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.752460 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.752393    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.753069 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.753041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7039a24-5b18-4989-a851-617b918efdd9-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.753172 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.753156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-config\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.753475 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.753460    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7039a24-5b18-4989-a851-617b918efdd9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.753892 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.753876    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7039a24-5b18-4989-a851-617b918efdd9-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.756432 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.756411    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc2r4\" (UniqueName: \"kubernetes.io/projected/d7039a24-5b18-4989-a851-617b918efdd9-kube-api-access-bc2r4\") pod \"prometheus-k8s-0\" (UID: \"d7039a24-5b18-4989-a851-617b918efdd9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:58.769080 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.769049    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f06bb59-c3f6-4e7f-984a-e64932b9cf59" path="/var/lib/kubelet/pods/4f06bb59-c3f6-4e7f-984a-e64932b9cf59/volumes"
Jul 14 11:20:58.872098 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:58.872003    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:20:59.005745 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:59.005663    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 11:20:59.007902 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:20:59.007878    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7039a24_5b18_4989_a851_617b918efdd9.slice/crio-44a04a16012438512ab8249da8c55a0aa73d4140342594115a4c1ebbb6224d03 WatchSource:0}: Error finding container 44a04a16012438512ab8249da8c55a0aa73d4140342594115a4c1ebbb6224d03: Status 404 returned error can't find the container with id 44a04a16012438512ab8249da8c55a0aa73d4140342594115a4c1ebbb6224d03
Jul 14 11:20:59.504493 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:59.504454    2564 generic.go:356] "Generic (PLEG): container finished" podID="d7039a24-5b18-4989-a851-617b918efdd9" containerID="5d8466214eb177199c42ab68f75c5dc0481656ab50416856222fda46538c0fee" exitCode=0
Jul 14 11:20:59.504940 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:59.504536    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerDied","Data":"5d8466214eb177199c42ab68f75c5dc0481656ab50416856222fda46538c0fee"}
Jul 14 11:20:59.504940 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:20:59.504640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerStarted","Data":"44a04a16012438512ab8249da8c55a0aa73d4140342594115a4c1ebbb6224d03"}
Jul 14 11:21:00.513086 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:00.513056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerStarted","Data":"5e49b6a032ca12240e4e57fef5e3630a55782e5134e6f064ddebecbe0aef8c5c"}
Jul 14 11:21:00.513443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:00.513118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerStarted","Data":"d3ffd29128fe3103693070c75281d5ad654704e874cf8188702a77785c5a991b"}
Jul 14 11:21:00.513443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:00.513144    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerStarted","Data":"93ff508cfa1fb0de31d0da3f0f72223cee12cac1c04271027ed30a69c532175e"}
Jul 14 11:21:00.513443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:00.513169    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerStarted","Data":"dbcabf5173d8335c29194e22535dc291bf6c3583722681a50c6ba8ccb76c15f1"}
Jul 14 11:21:00.513443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:00.513191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerStarted","Data":"5529617710fa8bbad6dd07f57abf0da17e09b9a384a3c806e1d0712138ce6527"}
Jul 14 11:21:00.513443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:00.513215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7039a24-5b18-4989-a851-617b918efdd9","Type":"ContainerStarted","Data":"fe150eab9c600cb0704106d2f7e25c91472a1915fb427639194f404f14ec3b78"}
Jul 14 11:21:00.541133 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:00.541082    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.522955118 podStartE2EDuration="2.54106449s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.109372ms" isStatefulPod=false podCreationTimestamp="2026-07-14 11:20:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:21:00.539248934 +0000 UTC m=+154.339804676" watchObservedRunningTime="2026-07-14 11:21:00.54106449 +0000 UTC m=+154.341620230"
Jul 14 11:21:03.873155 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:03.873119    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:21:58.872507 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:58.872471    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:21:58.887505 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:58.887480    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:21:59.749907 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:21:59.749854    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 11:23:26.577952 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:23:26.577924    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:23:26.582844 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:23:26.582818    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:23:26.583205 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:23:26.583189    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 11:24:01.475253 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.475217    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"]
Jul 14 11:24:01.477888 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.477869    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.480432 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.480410    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 14 11:24:01.480808 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.480791    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 14 11:24:01.480808 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.480801    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kb8tm\""
Jul 14 11:24:01.488379 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.488356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"]
Jul 14 11:24:01.597385 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.597353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d25d998-862d-426d-916b-fedbf8cc0cdd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d52zh\" (UID: \"2d25d998-862d-426d-916b-fedbf8cc0cdd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.597385 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.597388    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljxnh\" (UniqueName: \"kubernetes.io/projected/2d25d998-862d-426d-916b-fedbf8cc0cdd-kube-api-access-ljxnh\") pod \"openshift-lws-operator-fd8ccff4c-d52zh\" (UID: \"2d25d998-862d-426d-916b-fedbf8cc0cdd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.698659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.698612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d25d998-862d-426d-916b-fedbf8cc0cdd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d52zh\" (UID: \"2d25d998-862d-426d-916b-fedbf8cc0cdd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.698659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.698658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljxnh\" (UniqueName: \"kubernetes.io/projected/2d25d998-862d-426d-916b-fedbf8cc0cdd-kube-api-access-ljxnh\") pod \"openshift-lws-operator-fd8ccff4c-d52zh\" (UID: \"2d25d998-862d-426d-916b-fedbf8cc0cdd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.699130 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.699105    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d25d998-862d-426d-916b-fedbf8cc0cdd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-d52zh\" (UID: \"2d25d998-862d-426d-916b-fedbf8cc0cdd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.706854 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.706826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljxnh\" (UniqueName: \"kubernetes.io/projected/2d25d998-862d-426d-916b-fedbf8cc0cdd-kube-api-access-ljxnh\") pod \"openshift-lws-operator-fd8ccff4c-d52zh\" (UID: \"2d25d998-862d-426d-916b-fedbf8cc0cdd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.809434 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.809400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"
Jul 14 11:24:01.931462 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.931434    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh"]
Jul 14 11:24:01.934119 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:24:01.934094    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d25d998_862d_426d_916b_fedbf8cc0cdd.slice/crio-d6d14748e450e13d6139050aa645c85e8b4804c2e6efa766e54f266247e08275 WatchSource:0}: Error finding container d6d14748e450e13d6139050aa645c85e8b4804c2e6efa766e54f266247e08275: Status 404 returned error can't find the container with id d6d14748e450e13d6139050aa645c85e8b4804c2e6efa766e54f266247e08275
Jul 14 11:24:01.935589 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:01.935572    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:24:02.171345 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:02.171260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh" event={"ID":"2d25d998-862d-426d-916b-fedbf8cc0cdd","Type":"ContainerStarted","Data":"d6d14748e450e13d6139050aa645c85e8b4804c2e6efa766e54f266247e08275"}
Jul 14 11:24:05.186138 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:05.186097    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh" event={"ID":"2d25d998-862d-426d-916b-fedbf8cc0cdd","Type":"ContainerStarted","Data":"ff07d5abee1858c2bb001cbbc7f127f8c96230b56b3487ed92b99a82ba08388b"}
Jul 14 11:24:05.204301 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:05.204246    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-d52zh" podStartSLOduration=1.720384543 podStartE2EDuration="4.204231726s" totalImagesPullingTime="2.483847183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:05.203933191 +0000 UTC m=+339.004488934" watchObservedRunningTime="2026-07-14 11:24:05.204231726 +0000 UTC m=+339.004787465"
Jul 14 11:24:20.216340 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.216259    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"]
Jul 14 11:24:20.219275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.219259    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.222659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.222639    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 14 11:24:20.222802 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.222658    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 14 11:24:20.222802 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.222664    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-667wr\""
Jul 14 11:24:20.222802 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.222740    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 14 11:24:20.234180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.234156    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"]
Jul 14 11:24:20.244648 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.244615    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/243b252f-6f7c-4aac-afc8-180b8e7171fa-manager-config\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.244810 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.244679    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/243b252f-6f7c-4aac-afc8-180b8e7171fa-metrics-cert\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.244810 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.244763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243b252f-6f7c-4aac-afc8-180b8e7171fa-cert\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.244892 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.244808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vpcj\" (UniqueName: \"kubernetes.io/projected/243b252f-6f7c-4aac-afc8-180b8e7171fa-kube-api-access-9vpcj\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.345965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.345929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/243b252f-6f7c-4aac-afc8-180b8e7171fa-manager-config\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.345965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.345980    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/243b252f-6f7c-4aac-afc8-180b8e7171fa-metrics-cert\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.346234 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.346019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243b252f-6f7c-4aac-afc8-180b8e7171fa-cert\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.346234 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.346054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vpcj\" (UniqueName: \"kubernetes.io/projected/243b252f-6f7c-4aac-afc8-180b8e7171fa-kube-api-access-9vpcj\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.346606 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.346583    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/243b252f-6f7c-4aac-afc8-180b8e7171fa-manager-config\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.348599 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.348574    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/243b252f-6f7c-4aac-afc8-180b8e7171fa-metrics-cert\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.348722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.348617    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243b252f-6f7c-4aac-afc8-180b8e7171fa-cert\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.355235 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.355207    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vpcj\" (UniqueName: \"kubernetes.io/projected/243b252f-6f7c-4aac-afc8-180b8e7171fa-kube-api-access-9vpcj\") pod \"lws-controller-manager-985446bdb-8hzx9\" (UID: \"243b252f-6f7c-4aac-afc8-180b8e7171fa\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.528019 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.527985    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:20.658088 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:20.658062    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"]
Jul 14 11:24:20.660526 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:24:20.660484    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243b252f_6f7c_4aac_afc8_180b8e7171fa.slice/crio-4a0cb52d4fbff6b8c4c8bc76eefbf2695df325b175b59cbd17c46bf1a6b966ac WatchSource:0}: Error finding container 4a0cb52d4fbff6b8c4c8bc76eefbf2695df325b175b59cbd17c46bf1a6b966ac: Status 404 returned error can't find the container with id 4a0cb52d4fbff6b8c4c8bc76eefbf2695df325b175b59cbd17c46bf1a6b966ac
Jul 14 11:24:21.250034 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:21.250002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9" event={"ID":"243b252f-6f7c-4aac-afc8-180b8e7171fa","Type":"ContainerStarted","Data":"4a0cb52d4fbff6b8c4c8bc76eefbf2695df325b175b59cbd17c46bf1a6b966ac"}
Jul 14 11:24:23.260130 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:23.260034    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9" event={"ID":"243b252f-6f7c-4aac-afc8-180b8e7171fa","Type":"ContainerStarted","Data":"e50626b6076d348872657c500cb2659c5e09c7c8d3c893729dac503479526d06"}
Jul 14 11:24:23.260499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:23.260246    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:23.280623 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:23.280569    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9" podStartSLOduration=0.948959486 podStartE2EDuration="3.280556264s" totalImagesPullingTime="2.331596778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:23.279220563 +0000 UTC m=+357.079776326" watchObservedRunningTime="2026-07-14 11:24:23.280556264 +0000 UTC m=+357.081112004"
Jul 14 11:24:24.604344 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.604310    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"]
Jul 14 11:24:24.607656 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.607637    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.610429 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.610407    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gz7sx\""
Jul 14 11:24:24.610553 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.610451    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 11:24:24.610710 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.610675    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 11:24:24.610780 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.610682    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 11:24:24.610883 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.610870    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 11:24:24.628289 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.628265    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"]
Jul 14 11:24:24.683632 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.683596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e223985c-991b-4f3e-836f-8fbd992e4dea-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.683632 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.683634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e223985c-991b-4f3e-836f-8fbd992e4dea-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.683878 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.683660    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e223985c-991b-4f3e-836f-8fbd992e4dea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.683878 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.683726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e223985c-991b-4f3e-836f-8fbd992e4dea-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.683878 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.683750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb8rw\" (UniqueName: \"kubernetes.io/projected/e223985c-991b-4f3e-836f-8fbd992e4dea-kube-api-access-rb8rw\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.784913 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.784884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e223985c-991b-4f3e-836f-8fbd992e4dea-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.785103 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.784924    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e223985c-991b-4f3e-836f-8fbd992e4dea-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.785103 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.785042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e223985c-991b-4f3e-836f-8fbd992e4dea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.785103 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.785079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e223985c-991b-4f3e-836f-8fbd992e4dea-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.785103 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.785099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb8rw\" (UniqueName: \"kubernetes.io/projected/e223985c-991b-4f3e-836f-8fbd992e4dea-kube-api-access-rb8rw\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.785351 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.785331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e223985c-991b-4f3e-836f-8fbd992e4dea-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.787275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.787248    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e223985c-991b-4f3e-836f-8fbd992e4dea-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.787380 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.787352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e223985c-991b-4f3e-836f-8fbd992e4dea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.787380 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.787369    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e223985c-991b-4f3e-836f-8fbd992e4dea-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.793929 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.793905    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb8rw\" (UniqueName: \"kubernetes.io/projected/e223985c-991b-4f3e-836f-8fbd992e4dea-kube-api-access-rb8rw\") pod \"opendatahub-operator-controller-manager-555b6fb65-ld599\" (UID: \"e223985c-991b-4f3e-836f-8fbd992e4dea\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:24.917126 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:24.917047    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:25.054037 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:25.053989    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"]
Jul 14 11:24:25.056786 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:24:25.056749    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode223985c_991b_4f3e_836f_8fbd992e4dea.slice/crio-3113831c71c4c488ebf8bfddb659bec5bfaef3e1b68f68db51b3315cc927db11 WatchSource:0}: Error finding container 3113831c71c4c488ebf8bfddb659bec5bfaef3e1b68f68db51b3315cc927db11: Status 404 returned error can't find the container with id 3113831c71c4c488ebf8bfddb659bec5bfaef3e1b68f68db51b3315cc927db11
Jul 14 11:24:25.268885 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:25.268851    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599" event={"ID":"e223985c-991b-4f3e-836f-8fbd992e4dea","Type":"ContainerStarted","Data":"3113831c71c4c488ebf8bfddb659bec5bfaef3e1b68f68db51b3315cc927db11"}
Jul 14 11:24:28.284293 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:28.284245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599" event={"ID":"e223985c-991b-4f3e-836f-8fbd992e4dea","Type":"ContainerStarted","Data":"057417ea1a74dce63facbd64d90c8ab72525c61f768aa56693c5a9665545971e"}
Jul 14 11:24:29.290097 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:29.290058    2564 generic.go:356] "Generic (PLEG): container finished" podID="e223985c-991b-4f3e-836f-8fbd992e4dea" containerID="057417ea1a74dce63facbd64d90c8ab72525c61f768aa56693c5a9665545971e" exitCode=0
Jul 14 11:24:29.290097 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:29.290098    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599" event={"ID":"e223985c-991b-4f3e-836f-8fbd992e4dea","Type":"ContainerDied","Data":"057417ea1a74dce63facbd64d90c8ab72525c61f768aa56693c5a9665545971e"}
Jul 14 11:24:30.296658 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:30.296623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599" event={"ID":"e223985c-991b-4f3e-836f-8fbd992e4dea","Type":"ContainerStarted","Data":"c3e55c5739d18d8cc941079c1aab9c015081549c8ad9c6f6a1d3df2ed4e89abb"}
Jul 14 11:24:30.297063 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:30.296750    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:30.326611 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:30.326541    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599" podStartSLOduration=1.924362419 podStartE2EDuration="6.326525578s" totalImagesPullingTime="3.388655831s" totalInitContainerRuntime="1.013507328s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:30.325609755 +0000 UTC m=+364.126165505" watchObservedRunningTime="2026-07-14 11:24:30.326525578 +0000 UTC m=+364.127081318"
Jul 14 11:24:34.266497 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:34.266462    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-985446bdb-8hzx9"
Jul 14 11:24:41.303425 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:41.303394    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-ld599"
Jul 14 11:24:44.806737 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.806688    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"]
Jul 14 11:24:44.814471 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.814445    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:44.821688 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.821667    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 11:24:44.821849 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.821687    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tjbwp\""
Jul 14 11:24:44.822186 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.822165    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 11:24:44.822260 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.822192    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 11:24:44.822443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.822429    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 11:24:44.839277 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.839252    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"]
Jul 14 11:24:44.959671 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.959630    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/949e0818-e7ee-42df-9b5e-663f5527e81b-tls-certs\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:44.959849 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.959676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvpvl\" (UniqueName: \"kubernetes.io/projected/949e0818-e7ee-42df-9b5e-663f5527e81b-kube-api-access-tvpvl\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:44.959849 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:44.959758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/949e0818-e7ee-42df-9b5e-663f5527e81b-tmp\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.060470 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.060398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/949e0818-e7ee-42df-9b5e-663f5527e81b-tmp\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.060614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.060474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/949e0818-e7ee-42df-9b5e-663f5527e81b-tls-certs\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.060614 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.060494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpvl\" (UniqueName: \"kubernetes.io/projected/949e0818-e7ee-42df-9b5e-663f5527e81b-kube-api-access-tvpvl\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.060682 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:24:45.060609    2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 14 11:24:45.060731 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:24:45.060681    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/949e0818-e7ee-42df-9b5e-663f5527e81b-tls-certs podName:949e0818-e7ee-42df-9b5e-663f5527e81b nodeName:}" failed. No retries permitted until 2026-07-14 11:24:45.56066172 +0000 UTC m=+379.361217437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/949e0818-e7ee-42df-9b5e-663f5527e81b-tls-certs") pod "kube-auth-proxy-76f9775b55-p5kdl" (UID: "949e0818-e7ee-42df-9b5e-663f5527e81b") : secret "kube-auth-proxy-tls" not found
Jul 14 11:24:45.062673 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.062641    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/949e0818-e7ee-42df-9b5e-663f5527e81b-tmp\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.084822 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.084789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpvl\" (UniqueName: \"kubernetes.io/projected/949e0818-e7ee-42df-9b5e-663f5527e81b-kube-api-access-tvpvl\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.565037 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.565005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/949e0818-e7ee-42df-9b5e-663f5527e81b-tls-certs\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.567356 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.567327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/949e0818-e7ee-42df-9b5e-663f5527e81b-tls-certs\") pod \"kube-auth-proxy-76f9775b55-p5kdl\" (UID: \"949e0818-e7ee-42df-9b5e-663f5527e81b\") " pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.723904 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.723872    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"
Jul 14 11:24:45.922783 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:45.922687    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl"]
Jul 14 11:24:45.926227 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:24:45.926199    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949e0818_e7ee_42df_9b5e_663f5527e81b.slice/crio-addf2413409435c0e9280bb646dc5e9e35a7a0dd5661094e2fd811dfec24187f WatchSource:0}: Error finding container addf2413409435c0e9280bb646dc5e9e35a7a0dd5661094e2fd811dfec24187f: Status 404 returned error can't find the container with id addf2413409435c0e9280bb646dc5e9e35a7a0dd5661094e2fd811dfec24187f
Jul 14 11:24:46.361558 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:46.361521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl" event={"ID":"949e0818-e7ee-42df-9b5e-663f5527e81b","Type":"ContainerStarted","Data":"addf2413409435c0e9280bb646dc5e9e35a7a0dd5661094e2fd811dfec24187f"}
Jul 14 11:24:48.372777 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:48.372737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl" event={"ID":"949e0818-e7ee-42df-9b5e-663f5527e81b","Type":"ContainerStarted","Data":"049333844c6f6eb717aa5bf0929ca54af166c25425b5c111b7dd6b07fa85ef71"}
Jul 14 11:24:48.399810 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:24:48.399757    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76f9775b55-p5kdl" podStartSLOduration=2.065018466 podStartE2EDuration="4.399737052s" totalImagesPullingTime="2.334718586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:48.399049364 +0000 UTC m=+382.199605103" watchObservedRunningTime="2026-07-14 11:24:48.399737052 +0000 UTC m=+382.200292791"
Jul 14 11:26:26.394237 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.394202    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"]
Jul 14 11:26:26.396659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.396640    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"
Jul 14 11:26:26.400022 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.399999    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qsvhm\""
Jul 14 11:26:26.400148 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.400051    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 14 11:26:26.400148 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.400050    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 11:26:26.400604 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.400570    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 11:26:26.406069 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.406048    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"]
Jul 14 11:26:26.519997 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.519960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl4qh\" (UniqueName: \"kubernetes.io/projected/85c9751d-61bd-4072-b205-09b87b3ed292-kube-api-access-nl4qh\") pod \"dns-operator-controller-manager-864dd78c4c-qj8v7\" (UID: \"85c9751d-61bd-4072-b205-09b87b3ed292\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"
Jul 14 11:26:26.620420 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.620381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl4qh\" (UniqueName: \"kubernetes.io/projected/85c9751d-61bd-4072-b205-09b87b3ed292-kube-api-access-nl4qh\") pod \"dns-operator-controller-manager-864dd78c4c-qj8v7\" (UID: \"85c9751d-61bd-4072-b205-09b87b3ed292\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"
Jul 14 11:26:26.630381 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.630356    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 11:26:26.641190 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.641165    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 11:26:26.651918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.651869    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl4qh\" (UniqueName: \"kubernetes.io/projected/85c9751d-61bd-4072-b205-09b87b3ed292-kube-api-access-nl4qh\") pod \"dns-operator-controller-manager-864dd78c4c-qj8v7\" (UID: \"85c9751d-61bd-4072-b205-09b87b3ed292\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"
Jul 14 11:26:26.709820 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.709797    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qsvhm\""
Jul 14 11:26:26.717723 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.717681    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"
Jul 14 11:26:26.848686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:26.848662    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"]
Jul 14 11:26:26.850984 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:26:26.850952    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c9751d_61bd_4072_b205_09b87b3ed292.slice/crio-9b6c6c6ffc9949e1137c0d94bba64a74a66ac15e0241f9e17dfab116a7d52315 WatchSource:0}: Error finding container 9b6c6c6ffc9949e1137c0d94bba64a74a66ac15e0241f9e17dfab116a7d52315: Status 404 returned error can't find the container with id 9b6c6c6ffc9949e1137c0d94bba64a74a66ac15e0241f9e17dfab116a7d52315
Jul 14 11:26:27.619323 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.619292    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"]
Jul 14 11:26:27.622712 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.622408    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.624880 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.624835    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 14 11:26:27.625405 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.625375    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 14 11:26:27.625796 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.625768    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-82t9v\""
Jul 14 11:26:27.635664 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.635639    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"]
Jul 14 11:26:27.729032 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.729001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.729201 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.729046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s6qw\" (UniqueName: \"kubernetes.io/projected/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-kube-api-access-2s6qw\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.729201 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.729117    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.769865 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.769834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7" event={"ID":"85c9751d-61bd-4072-b205-09b87b3ed292","Type":"ContainerStarted","Data":"9b6c6c6ffc9949e1137c0d94bba64a74a66ac15e0241f9e17dfab116a7d52315"}
Jul 14 11:26:27.830099 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.830018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s6qw\" (UniqueName: \"kubernetes.io/projected/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-kube-api-access-2s6qw\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.830099 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.830097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.830332 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.830185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.830445 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:26:27.830403    2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 14 11:26:27.830511 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:26:27.830486    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-plugin-serving-cert podName:94f17e02-1ba9-4eae-aea7-fd0c2596c6f5 nodeName:}" failed. No retries permitted until 2026-07-14 11:26:28.330462218 +0000 UTC m=+482.131017937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-td7qd" (UID: "94f17e02-1ba9-4eae-aea7-fd0c2596c6f5") : secret "plugin-serving-cert" not found
Jul 14 11:26:27.830912 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.830890    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:27.855570 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:27.855544    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s6qw\" (UniqueName: \"kubernetes.io/projected/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-kube-api-access-2s6qw\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:28.335883 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:28.335843    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:28.338188 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:28.338162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94f17e02-1ba9-4eae-aea7-fd0c2596c6f5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-td7qd\" (UID: \"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:28.534580 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:28.534545    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"
Jul 14 11:26:28.674208 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:28.674177    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd"]
Jul 14 11:26:28.884001 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:26:28.883928    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94f17e02_1ba9_4eae_aea7_fd0c2596c6f5.slice/crio-b2fd80f0aa7d590b06e66a556be50effe3603651ada27e835427695826e1b18c WatchSource:0}: Error finding container b2fd80f0aa7d590b06e66a556be50effe3603651ada27e835427695826e1b18c: Status 404 returned error can't find the container with id b2fd80f0aa7d590b06e66a556be50effe3603651ada27e835427695826e1b18c
Jul 14 11:26:28.914430 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:28.914409    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 14 11:26:29.780686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:29.780652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7" event={"ID":"85c9751d-61bd-4072-b205-09b87b3ed292","Type":"ContainerStarted","Data":"cf59a1750b56c7d171e33ccf6c05a100ea4f603779989043052155be30c00799"}
Jul 14 11:26:29.781113 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:29.780752    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"
Jul 14 11:26:29.781826 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:29.781806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd" event={"ID":"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5","Type":"ContainerStarted","Data":"b2fd80f0aa7d590b06e66a556be50effe3603651ada27e835427695826e1b18c"}
Jul 14 11:26:40.789952 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:40.789691    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7"
Jul 14 11:26:40.818366 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:40.818309    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj8v7" podStartSLOduration=12.759907298 podStartE2EDuration="14.818287081s" totalImagesPullingTime="2.058379783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:29.797403732 +0000 UTC m=+483.597959472" watchObservedRunningTime="2026-07-14 11:26:40.818287081 +0000 UTC m=+494.618842822"
Jul 14 11:26:51.894775 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:51.894733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd" event={"ID":"94f17e02-1ba9-4eae-aea7-fd0c2596c6f5","Type":"ContainerStarted","Data":"cc149dc0e5f16e747553a50e159f73a95e9ed0bb14f4820ac24072bbeeeb8e6a"}
Jul 14 11:26:51.917959 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:26:51.917900    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-td7qd" podStartSLOduration=2.758468625 podStartE2EDuration="24.917867699s" totalImagesPullingTime="22.159399074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:51.917289478 +0000 UTC m=+505.717845217" watchObservedRunningTime="2026-07-14 11:26:51.917867699 +0000 UTC m=+505.718423440"
Jul 14 11:27:11.239970 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.239935    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:27:11.257496 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.257468    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:27:11.257653 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.257576    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.259885 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.259864    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 11:27:11.289650 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.289624    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:27:11.406028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.405992    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfl5h\" (UniqueName: \"kubernetes.io/projected/e2847ace-c277-499f-83e2-6e3e24e83cdb-kube-api-access-sfl5h\") pod \"limitador-limitador-54d6d988d5-b6pjt\" (UID: \"e2847ace-c277-499f-83e2-6e3e24e83cdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.406185 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.406039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e2847ace-c277-499f-83e2-6e3e24e83cdb-config-file\") pod \"limitador-limitador-54d6d988d5-b6pjt\" (UID: \"e2847ace-c277-499f-83e2-6e3e24e83cdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.507301 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.507210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e2847ace-c277-499f-83e2-6e3e24e83cdb-config-file\") pod \"limitador-limitador-54d6d988d5-b6pjt\" (UID: \"e2847ace-c277-499f-83e2-6e3e24e83cdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.507465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.507311    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfl5h\" (UniqueName: \"kubernetes.io/projected/e2847ace-c277-499f-83e2-6e3e24e83cdb-kube-api-access-sfl5h\") pod \"limitador-limitador-54d6d988d5-b6pjt\" (UID: \"e2847ace-c277-499f-83e2-6e3e24e83cdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.507877 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.507855    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e2847ace-c277-499f-83e2-6e3e24e83cdb-config-file\") pod \"limitador-limitador-54d6d988d5-b6pjt\" (UID: \"e2847ace-c277-499f-83e2-6e3e24e83cdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.516119 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.516101    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfl5h\" (UniqueName: \"kubernetes.io/projected/e2847ace-c277-499f-83e2-6e3e24e83cdb-kube-api-access-sfl5h\") pod \"limitador-limitador-54d6d988d5-b6pjt\" (UID: \"e2847ace-c277-499f-83e2-6e3e24e83cdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.567813 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.567775    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:11.696999 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.696977    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:27:11.699544 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:27:11.699507    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2847ace_c277_499f_83e2_6e3e24e83cdb.slice/crio-a4cac8bb0e99fdfb63e5e28bb848ea9f4d65d57ab57bc227c5394aa995b82730 WatchSource:0}: Error finding container a4cac8bb0e99fdfb63e5e28bb848ea9f4d65d57ab57bc227c5394aa995b82730: Status 404 returned error can't find the container with id a4cac8bb0e99fdfb63e5e28bb848ea9f4d65d57ab57bc227c5394aa995b82730
Jul 14 11:27:11.708771 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.708744    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-784lp"]
Jul 14 11:27:11.711577 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.711559    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-784lp"
Jul 14 11:27:11.713842 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.713823    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-s64ln\""
Jul 14 11:27:11.719390 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.719368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-784lp"]
Jul 14 11:27:11.810514 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.810480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4lkk\" (UniqueName: \"kubernetes.io/projected/c0678287-739d-4e2b-b69a-c008f92122ba-kube-api-access-z4lkk\") pod \"authorino-58cd955675-784lp\" (UID: \"c0678287-739d-4e2b-b69a-c008f92122ba\") " pod="kuadrant-system/authorino-58cd955675-784lp"
Jul 14 11:27:11.911458 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.911418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4lkk\" (UniqueName: \"kubernetes.io/projected/c0678287-739d-4e2b-b69a-c008f92122ba-kube-api-access-z4lkk\") pod \"authorino-58cd955675-784lp\" (UID: \"c0678287-739d-4e2b-b69a-c008f92122ba\") " pod="kuadrant-system/authorino-58cd955675-784lp"
Jul 14 11:27:11.920910 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.920887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4lkk\" (UniqueName: \"kubernetes.io/projected/c0678287-739d-4e2b-b69a-c008f92122ba-kube-api-access-z4lkk\") pod \"authorino-58cd955675-784lp\" (UID: \"c0678287-739d-4e2b-b69a-c008f92122ba\") " pod="kuadrant-system/authorino-58cd955675-784lp"
Jul 14 11:27:11.985779 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:11.985743    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt" event={"ID":"e2847ace-c277-499f-83e2-6e3e24e83cdb","Type":"ContainerStarted","Data":"a4cac8bb0e99fdfb63e5e28bb848ea9f4d65d57ab57bc227c5394aa995b82730"}
Jul 14 11:27:12.021225 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:12.021190    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-784lp"
Jul 14 11:27:12.159965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:12.159928    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-784lp"]
Jul 14 11:27:12.162548 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:27:12.162517    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0678287_739d_4e2b_b69a_c008f92122ba.slice/crio-a725a0a57a39b39f880078d58c84997f78e056fe42f6365140bb3478f13b29f7 WatchSource:0}: Error finding container a725a0a57a39b39f880078d58c84997f78e056fe42f6365140bb3478f13b29f7: Status 404 returned error can't find the container with id a725a0a57a39b39f880078d58c84997f78e056fe42f6365140bb3478f13b29f7
Jul 14 11:27:12.992385 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:12.992352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-784lp" event={"ID":"c0678287-739d-4e2b-b69a-c008f92122ba","Type":"ContainerStarted","Data":"a725a0a57a39b39f880078d58c84997f78e056fe42f6365140bb3478f13b29f7"}
Jul 14 11:27:15.006039 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:15.005989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt" event={"ID":"e2847ace-c277-499f-83e2-6e3e24e83cdb","Type":"ContainerStarted","Data":"0d40fdecfec463439cf5d91e96e368066b0f323f61f9ff568778c65bfd18f6aa"}
Jul 14 11:27:15.006543 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:15.006250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:17.017322 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:17.017283    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-784lp" event={"ID":"c0678287-739d-4e2b-b69a-c008f92122ba","Type":"ContainerStarted","Data":"b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6"}
Jul 14 11:27:17.031880 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:17.031832    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt" podStartSLOduration=3.5078564930000002 podStartE2EDuration="6.031812115s" totalImagesPullingTime="2.523955622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:15.03426058 +0000 UTC m=+528.834816321" watchObservedRunningTime="2026-07-14 11:27:17.031812115 +0000 UTC m=+530.832367860"
Jul 14 11:27:17.033057 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:17.033019    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-784lp" podStartSLOduration=1.6634593500000001 podStartE2EDuration="6.0330064s" totalImagesPullingTime="4.36954705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:17.03140709 +0000 UTC m=+530.831962829" watchObservedRunningTime="2026-07-14 11:27:17.0330064 +0000 UTC m=+530.833562140"
Jul 14 11:27:17.247202 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:17.247170    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-784lp"]
Jul 14 11:27:19.026319 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:19.026234    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-784lp" podUID="c0678287-739d-4e2b-b69a-c008f92122ba" containerName="authorino" containerID="cri-o://b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6" gracePeriod=30
Jul 14 11:27:19.316449 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:19.316413    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-784lp"
Jul 14 11:27:19.480221 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:19.480176    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4lkk\" (UniqueName: \"kubernetes.io/projected/c0678287-739d-4e2b-b69a-c008f92122ba-kube-api-access-z4lkk\") pod \"c0678287-739d-4e2b-b69a-c008f92122ba\" (UID: \"c0678287-739d-4e2b-b69a-c008f92122ba\") "
Jul 14 11:27:19.482351 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:19.482315    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0678287-739d-4e2b-b69a-c008f92122ba-kube-api-access-z4lkk" (OuterVolumeSpecName: "kube-api-access-z4lkk") pod "c0678287-739d-4e2b-b69a-c008f92122ba" (UID: "c0678287-739d-4e2b-b69a-c008f92122ba"). InnerVolumeSpecName "kube-api-access-z4lkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:27:19.581255 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:19.581171    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4lkk\" (UniqueName: \"kubernetes.io/projected/c0678287-739d-4e2b-b69a-c008f92122ba-kube-api-access-z4lkk\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:27:20.032091 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.032055    2564 generic.go:356] "Generic (PLEG): container finished" podID="c0678287-739d-4e2b-b69a-c008f92122ba" containerID="b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6" exitCode=0
Jul 14 11:27:20.032510 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.032101    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-784lp"
Jul 14 11:27:20.032510 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.032147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-784lp" event={"ID":"c0678287-739d-4e2b-b69a-c008f92122ba","Type":"ContainerDied","Data":"b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6"}
Jul 14 11:27:20.032510 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.032196    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-784lp" event={"ID":"c0678287-739d-4e2b-b69a-c008f92122ba","Type":"ContainerDied","Data":"a725a0a57a39b39f880078d58c84997f78e056fe42f6365140bb3478f13b29f7"}
Jul 14 11:27:20.032510 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.032214    2564 scope.go:117] "RemoveContainer" containerID="b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6"
Jul 14 11:27:20.040526 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.040510    2564 scope.go:117] "RemoveContainer" containerID="b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6"
Jul 14 11:27:20.040810 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:27:20.040792    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6\": container with ID starting with b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6 not found: ID does not exist" containerID="b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6"
Jul 14 11:27:20.040884 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.040816    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6"} err="failed to get container status \"b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6\": rpc error: code = NotFound desc = could not find container \"b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6\": container with ID starting with b4f69f03fdefc6d29ee653b05a8e19807df7b476dfe85dbd9d5852241c2bbaa6 not found: ID does not exist"
Jul 14 11:27:20.051072 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.051051    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-784lp"]
Jul 14 11:27:20.054916 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.054896    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-784lp"]
Jul 14 11:27:20.768770 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:20.768739    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0678287-739d-4e2b-b69a-c008f92122ba" path="/var/lib/kubelet/pods/c0678287-739d-4e2b-b69a-c008f92122ba/volumes"
Jul 14 11:27:26.012387 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:26.012358    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"
Jul 14 11:27:39.908018 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.907980    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mgs9x"]
Jul 14 11:27:39.908460 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.908306    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0678287-739d-4e2b-b69a-c008f92122ba" containerName="authorino"
Jul 14 11:27:39.908460 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.908318    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0678287-739d-4e2b-b69a-c008f92122ba" containerName="authorino"
Jul 14 11:27:39.908460 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.908381    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0678287-739d-4e2b-b69a-c008f92122ba" containerName="authorino"
Jul 14 11:27:39.936492 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.936465    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mgs9x"]
Jul 14 11:27:39.936639 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.936575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mgs9x"
Jul 14 11:27:39.939044 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.939022    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-s64ln\""
Jul 14 11:27:39.949016 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:39.948997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbclh\" (UniqueName: \"kubernetes.io/projected/77028d4f-7e95-4fe3-96b3-94d7e3230bf3-kube-api-access-sbclh\") pod \"authorino-5d4b67c446-mgs9x\" (UID: \"77028d4f-7e95-4fe3-96b3-94d7e3230bf3\") " pod="kuadrant-system/authorino-5d4b67c446-mgs9x"
Jul 14 11:27:40.050322 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.050292    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbclh\" (UniqueName: \"kubernetes.io/projected/77028d4f-7e95-4fe3-96b3-94d7e3230bf3-kube-api-access-sbclh\") pod \"authorino-5d4b67c446-mgs9x\" (UID: \"77028d4f-7e95-4fe3-96b3-94d7e3230bf3\") " pod="kuadrant-system/authorino-5d4b67c446-mgs9x"
Jul 14 11:27:40.058528 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.058499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbclh\" (UniqueName: \"kubernetes.io/projected/77028d4f-7e95-4fe3-96b3-94d7e3230bf3-kube-api-access-sbclh\") pod \"authorino-5d4b67c446-mgs9x\" (UID: \"77028d4f-7e95-4fe3-96b3-94d7e3230bf3\") " pod="kuadrant-system/authorino-5d4b67c446-mgs9x"
Jul 14 11:27:40.163946 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.163868    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mgs9x"]
Jul 14 11:27:40.164075 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.164065    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mgs9x"
Jul 14 11:27:40.192615 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.192588    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-8sxwg"]
Jul 14 11:27:40.196118 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.196101    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:40.198144 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.198125    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 14 11:27:40.204132 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.204107    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-8sxwg"]
Jul 14 11:27:40.233042 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.233017    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-8sxwg"]
Jul 14 11:27:40.233282 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:27:40.233261    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-kcjrq tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-kcjrq tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-8sxwg" podUID="a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4"
Jul 14 11:27:40.252206 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.252171    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcjrq\" (UniqueName: \"kubernetes.io/projected/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-kube-api-access-kcjrq\") pod \"authorino-76877994-8sxwg\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") " pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:40.252324 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.252216    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-tls-cert\") pod \"authorino-76877994-8sxwg\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") " pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:40.265927 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.265894    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bdcd6b6f6-g58x5"]
Jul 14 11:27:40.269509 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.269477    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.276198 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.276158    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bdcd6b6f6-g58x5"]
Jul 14 11:27:40.294114 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.294089    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mgs9x"]
Jul 14 11:27:40.294623 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:27:40.294597    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77028d4f_7e95_4fe3_96b3_94d7e3230bf3.slice/crio-97470d049635ac210b2005d99544dbb9e2c7eb8742b25e29cbc8c37efe24f9a1 WatchSource:0}: Error finding container 97470d049635ac210b2005d99544dbb9e2c7eb8742b25e29cbc8c37efe24f9a1: Status 404 returned error can't find the container with id 97470d049635ac210b2005d99544dbb9e2c7eb8742b25e29cbc8c37efe24f9a1
Jul 14 11:27:40.352974 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.352944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9hjx\" (UniqueName: \"kubernetes.io/projected/42534c61-e8af-4a4c-b03f-290b9efa1255-kube-api-access-p9hjx\") pod \"authorino-7bdcd6b6f6-g58x5\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") " pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.353110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.352983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42534c61-e8af-4a4c-b03f-290b9efa1255-tls-cert\") pod \"authorino-7bdcd6b6f6-g58x5\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") " pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.353110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.353033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcjrq\" (UniqueName: \"kubernetes.io/projected/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-kube-api-access-kcjrq\") pod \"authorino-76877994-8sxwg\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") " pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:40.353110 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.353052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-tls-cert\") pod \"authorino-76877994-8sxwg\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") " pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:40.355503 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.355479    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-tls-cert\") pod \"authorino-76877994-8sxwg\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") " pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:40.360610 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.360585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcjrq\" (UniqueName: \"kubernetes.io/projected/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-kube-api-access-kcjrq\") pod \"authorino-76877994-8sxwg\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") " pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:40.454161 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.454079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42534c61-e8af-4a4c-b03f-290b9efa1255-tls-cert\") pod \"authorino-7bdcd6b6f6-g58x5\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") " pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.454294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.454196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9hjx\" (UniqueName: \"kubernetes.io/projected/42534c61-e8af-4a4c-b03f-290b9efa1255-kube-api-access-p9hjx\") pod \"authorino-7bdcd6b6f6-g58x5\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") " pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.456284 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.456267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42534c61-e8af-4a4c-b03f-290b9efa1255-tls-cert\") pod \"authorino-7bdcd6b6f6-g58x5\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") " pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.462239 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.462223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9hjx\" (UniqueName: \"kubernetes.io/projected/42534c61-e8af-4a4c-b03f-290b9efa1255-kube-api-access-p9hjx\") pod \"authorino-7bdcd6b6f6-g58x5\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") " pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.580254 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.580220    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:27:40.708301 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:40.708280    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bdcd6b6f6-g58x5"]
Jul 14 11:27:40.711079 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:27:40.711051    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42534c61_e8af_4a4c_b03f_290b9efa1255.slice/crio-21fe1ddd948331c67bf89d9e4337b75fc12e92a4d03c6a4d39d5e290fe91311a WatchSource:0}: Error finding container 21fe1ddd948331c67bf89d9e4337b75fc12e92a4d03c6a4d39d5e290fe91311a: Status 404 returned error can't find the container with id 21fe1ddd948331c67bf89d9e4337b75fc12e92a4d03c6a4d39d5e290fe91311a
Jul 14 11:27:41.125076 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.125012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mgs9x" event={"ID":"77028d4f-7e95-4fe3-96b3-94d7e3230bf3","Type":"ContainerStarted","Data":"31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c"}
Jul 14 11:27:41.125076 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.125059    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mgs9x" event={"ID":"77028d4f-7e95-4fe3-96b3-94d7e3230bf3","Type":"ContainerStarted","Data":"97470d049635ac210b2005d99544dbb9e2c7eb8742b25e29cbc8c37efe24f9a1"}
Jul 14 11:27:41.125076 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.125022    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-mgs9x" podUID="77028d4f-7e95-4fe3-96b3-94d7e3230bf3" containerName="authorino" containerID="cri-o://31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c" gracePeriod=30
Jul 14 11:27:41.126448 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.126424    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:41.126536 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.126426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5" event={"ID":"42534c61-e8af-4a4c-b03f-290b9efa1255","Type":"ContainerStarted","Data":"21fe1ddd948331c67bf89d9e4337b75fc12e92a4d03c6a4d39d5e290fe91311a"}
Jul 14 11:27:41.140239 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.140075    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-mgs9x" podStartSLOduration=1.768850997 podStartE2EDuration="2.14005758s" totalImagesPullingTime="371.206583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:41.139633366 +0000 UTC m=+554.940189108" watchObservedRunningTime="2026-07-14 11:27:41.14005758 +0000 UTC m=+554.940613321"
Jul 14 11:27:41.212731 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.212690    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:41.260265 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.260237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-tls-cert\") pod \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") "
Jul 14 11:27:41.260422 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.260301    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcjrq\" (UniqueName: \"kubernetes.io/projected/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-kube-api-access-kcjrq\") pod \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\" (UID: \"a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4\") "
Jul 14 11:27:41.262218 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.262189    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4" (UID: "a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:27:41.262331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.262309    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-kube-api-access-kcjrq" (OuterVolumeSpecName: "kube-api-access-kcjrq") pod "a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4" (UID: "a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4"). InnerVolumeSpecName "kube-api-access-kcjrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:27:41.360885 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.360861    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-tls-cert\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:27:41.360885 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.360885    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcjrq\" (UniqueName: \"kubernetes.io/projected/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4-kube-api-access-kcjrq\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:27:41.363687 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.363667    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mgs9x"
Jul 14 11:27:41.461530 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.461501    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbclh\" (UniqueName: \"kubernetes.io/projected/77028d4f-7e95-4fe3-96b3-94d7e3230bf3-kube-api-access-sbclh\") pod \"77028d4f-7e95-4fe3-96b3-94d7e3230bf3\" (UID: \"77028d4f-7e95-4fe3-96b3-94d7e3230bf3\") "
Jul 14 11:27:41.463493 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.463462    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77028d4f-7e95-4fe3-96b3-94d7e3230bf3-kube-api-access-sbclh" (OuterVolumeSpecName: "kube-api-access-sbclh") pod "77028d4f-7e95-4fe3-96b3-94d7e3230bf3" (UID: "77028d4f-7e95-4fe3-96b3-94d7e3230bf3"). InnerVolumeSpecName "kube-api-access-sbclh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:27:41.562671 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:41.562639    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbclh\" (UniqueName: \"kubernetes.io/projected/77028d4f-7e95-4fe3-96b3-94d7e3230bf3-kube-api-access-sbclh\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:27:42.131922 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.131878    2564 generic.go:356] "Generic (PLEG): container finished" podID="77028d4f-7e95-4fe3-96b3-94d7e3230bf3" containerID="31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c" exitCode=0
Jul 14 11:27:42.131922 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.131919    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mgs9x"
Jul 14 11:27:42.132355 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.131967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mgs9x" event={"ID":"77028d4f-7e95-4fe3-96b3-94d7e3230bf3","Type":"ContainerDied","Data":"31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c"}
Jul 14 11:27:42.132355 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.132012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mgs9x" event={"ID":"77028d4f-7e95-4fe3-96b3-94d7e3230bf3","Type":"ContainerDied","Data":"97470d049635ac210b2005d99544dbb9e2c7eb8742b25e29cbc8c37efe24f9a1"}
Jul 14 11:27:42.132355 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.132032    2564 scope.go:117] "RemoveContainer" containerID="31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c"
Jul 14 11:27:42.133679 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.133660    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-8sxwg"
Jul 14 11:27:42.133819 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.133689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5" event={"ID":"42534c61-e8af-4a4c-b03f-290b9efa1255","Type":"ContainerStarted","Data":"e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31"}
Jul 14 11:27:42.140569 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.140551    2564 scope.go:117] "RemoveContainer" containerID="31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c"
Jul 14 11:27:42.140862 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:27:42.140843    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c\": container with ID starting with 31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c not found: ID does not exist" containerID="31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c"
Jul 14 11:27:42.140922 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.140873    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c"} err="failed to get container status \"31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c\": rpc error: code = NotFound desc = could not find container \"31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c\": container with ID starting with 31bf8e28bb79678dc0938aa5b2407b576756f7d1eae00e6c208c2dcb1268976c not found: ID does not exist"
Jul 14 11:27:42.152977 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.152938    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5" podStartSLOduration=1.7645772979999998 podStartE2EDuration="2.15292545s" totalImagesPullingTime="388.348152ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:42.151550498 +0000 UTC m=+555.952106239" watchObservedRunningTime="2026-07-14 11:27:42.15292545 +0000 UTC m=+555.953481190"
Jul 14 11:27:42.166149 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.166123    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mgs9x"]
Jul 14 11:27:42.173295 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.173272    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mgs9x"]
Jul 14 11:27:42.199374 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.199348    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-8sxwg"]
Jul 14 11:27:42.209733 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.209687    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-8sxwg"]
Jul 14 11:27:42.769855 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.769824    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77028d4f-7e95-4fe3-96b3-94d7e3230bf3" path="/var/lib/kubelet/pods/77028d4f-7e95-4fe3-96b3-94d7e3230bf3/volumes"
Jul 14 11:27:42.770181 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:27:42.770169    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4" path="/var/lib/kubelet/pods/a3b0b38a-ecb5-4c33-baf8-e7c4ee3a96c4/volumes"
Jul 14 11:28:20.102237 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:28:20.102201    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:28:26.613113 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:28:26.613076    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:28:26.614578 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:28:26.614558    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:28:55.761499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:28:55.761408    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:29:12.969990 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:12.969953    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:29:18.770618 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:18.770585    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:29:29.965579 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:29.965543    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:29:39.870850 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:39.870816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:29:45.570036 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.570004    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"]
Jul 14 11:29:45.570390 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.570364    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77028d4f-7e95-4fe3-96b3-94d7e3230bf3" containerName="authorino"
Jul 14 11:29:45.570390 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.570378    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77028d4f-7e95-4fe3-96b3-94d7e3230bf3" containerName="authorino"
Jul 14 11:29:45.570463 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.570438    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77028d4f-7e95-4fe3-96b3-94d7e3230bf3" containerName="authorino"
Jul 14 11:29:45.572540 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.572516    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.575756 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.575733    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vkrbv\""
Jul 14 11:29:45.575877 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.575783    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 11:29:45.575877 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.575758    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 14 11:29:45.575998 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.575783    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 11:29:45.581636 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.581616    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"]
Jul 14 11:29:45.728165 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.728131    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.728165 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.728163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.728369 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.728188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.728369 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.728260    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/505e78a2-7e28-4c92-b70d-262066e2e270-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.728369 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.728284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ttz\" (UniqueName: \"kubernetes.io/projected/505e78a2-7e28-4c92-b70d-262066e2e270-kube-api-access-m6ttz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.728369 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.728319    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.828908 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.828816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.828908 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.828863    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.828908 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.828896    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.829184 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.829048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/505e78a2-7e28-4c92-b70d-262066e2e270-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.829184 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.829085    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ttz\" (UniqueName: \"kubernetes.io/projected/505e78a2-7e28-4c92-b70d-262066e2e270-kube-api-access-m6ttz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.829184 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.829127    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.829184 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.829137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.829381 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.829268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.829452 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.829431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.831088 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.831057    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/505e78a2-7e28-4c92-b70d-262066e2e270-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.831558 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.831535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/505e78a2-7e28-4c92-b70d-262066e2e270-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.837806 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.837783    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ttz\" (UniqueName: \"kubernetes.io/projected/505e78a2-7e28-4c92-b70d-262066e2e270-kube-api-access-m6ttz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9\" (UID: \"505e78a2-7e28-4c92-b70d-262066e2e270\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:45.882165 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:45.882130    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:46.010948 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:46.010923    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"]
Jul 14 11:29:46.013312 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:29:46.013283    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod505e78a2_7e28_4c92_b70d_262066e2e270.slice/crio-6a9343b1f4a38b45d86741ed6b9b6cc03d7a6b715f72466cd2c0e89032621862 WatchSource:0}: Error finding container 6a9343b1f4a38b45d86741ed6b9b6cc03d7a6b715f72466cd2c0e89032621862: Status 404 returned error can't find the container with id 6a9343b1f4a38b45d86741ed6b9b6cc03d7a6b715f72466cd2c0e89032621862
Jul 14 11:29:46.014750 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:46.014734    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:29:46.674468 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:46.674426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9" event={"ID":"505e78a2-7e28-4c92-b70d-262066e2e270","Type":"ContainerStarted","Data":"6a9343b1f4a38b45d86741ed6b9b6cc03d7a6b715f72466cd2c0e89032621862"}
Jul 14 11:29:47.680910 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:47.680875    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9" event={"ID":"505e78a2-7e28-4c92-b70d-262066e2e270","Type":"ContainerStarted","Data":"64ad79dbf3c65abccca981ab9dc3a587ddc1a0d9f377d8a4f76e307e8a07f54e"}
Jul 14 11:29:48.976977 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:48.976904    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9" podStartSLOduration=2.8483344710000003 podStartE2EDuration="3.976880556s" totalImagesPullingTime="1.128546085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:29:47.697787935 +0000 UTC m=+681.498343688" watchObservedRunningTime="2026-07-14 11:29:48.976880556 +0000 UTC m=+682.777436298"
Jul 14 11:29:48.977403 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:48.977359    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:29:55.882724 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:55.882660    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:55.883179 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:55.882762    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:55.895489 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:55.895468    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:29:56.734893 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:29:56.734867    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9"
Jul 14 11:30:26.523481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.523445    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-97f67d9-g9wmz"]
Jul 14 11:30:26.526932 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.526910    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.533396 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.533369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-97f67d9-g9wmz"]
Jul 14 11:30:26.582322 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.582294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97zx7\" (UniqueName: \"kubernetes.io/projected/52f31154-5875-44bc-86e9-514da0d7db6f-kube-api-access-97zx7\") pod \"authorino-97f67d9-g9wmz\" (UID: \"52f31154-5875-44bc-86e9-514da0d7db6f\") " pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.582433 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.582341    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f31154-5875-44bc-86e9-514da0d7db6f-tls-cert\") pod \"authorino-97f67d9-g9wmz\" (UID: \"52f31154-5875-44bc-86e9-514da0d7db6f\") " pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.683482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.683453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f31154-5875-44bc-86e9-514da0d7db6f-tls-cert\") pod \"authorino-97f67d9-g9wmz\" (UID: \"52f31154-5875-44bc-86e9-514da0d7db6f\") " pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.683640 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.683516    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97zx7\" (UniqueName: \"kubernetes.io/projected/52f31154-5875-44bc-86e9-514da0d7db6f-kube-api-access-97zx7\") pod \"authorino-97f67d9-g9wmz\" (UID: \"52f31154-5875-44bc-86e9-514da0d7db6f\") " pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.685908 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.685877    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/52f31154-5875-44bc-86e9-514da0d7db6f-tls-cert\") pod \"authorino-97f67d9-g9wmz\" (UID: \"52f31154-5875-44bc-86e9-514da0d7db6f\") " pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.691326 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.691301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97zx7\" (UniqueName: \"kubernetes.io/projected/52f31154-5875-44bc-86e9-514da0d7db6f-kube-api-access-97zx7\") pod \"authorino-97f67d9-g9wmz\" (UID: \"52f31154-5875-44bc-86e9-514da0d7db6f\") " pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.835962 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.835889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-97f67d9-g9wmz"
Jul 14 11:30:26.961571 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:26.961136    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-97f67d9-g9wmz"]
Jul 14 11:30:26.963413 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:30:26.963377    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52f31154_5875_44bc_86e9_514da0d7db6f.slice/crio-0ee97f35e72aaa293f8c4172bb6dd7b9228c8674fd9c06830e2fbe373476d90d WatchSource:0}: Error finding container 0ee97f35e72aaa293f8c4172bb6dd7b9228c8674fd9c06830e2fbe373476d90d: Status 404 returned error can't find the container with id 0ee97f35e72aaa293f8c4172bb6dd7b9228c8674fd9c06830e2fbe373476d90d
Jul 14 11:30:27.857962 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:27.857915    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-97f67d9-g9wmz" event={"ID":"52f31154-5875-44bc-86e9-514da0d7db6f","Type":"ContainerStarted","Data":"b8124c62eedcc2e8fe4352de5fea757524f65bc1491521082ffb4eec4b1d1f24"}
Jul 14 11:30:27.858384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:27.857973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-97f67d9-g9wmz" event={"ID":"52f31154-5875-44bc-86e9-514da0d7db6f","Type":"ContainerStarted","Data":"0ee97f35e72aaa293f8c4172bb6dd7b9228c8674fd9c06830e2fbe373476d90d"}
Jul 14 11:30:27.878021 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:27.877960    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-97f67d9-g9wmz" podStartSLOduration=1.4922338769999999 podStartE2EDuration="1.87793992s" totalImagesPullingTime="385.706043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:30:27.87572248 +0000 UTC m=+721.676278220" watchObservedRunningTime="2026-07-14 11:30:27.87793992 +0000 UTC m=+721.678495659"
Jul 14 11:30:27.909920 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:27.909882    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bdcd6b6f6-g58x5"]
Jul 14 11:30:27.910133 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:27.910105    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5" podUID="42534c61-e8af-4a4c-b03f-290b9efa1255" containerName="authorino" containerID="cri-o://e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31" gracePeriod=30
Jul 14 11:30:28.179476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.179446    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:30:28.298481 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.298451    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42534c61-e8af-4a4c-b03f-290b9efa1255-tls-cert\") pod \"42534c61-e8af-4a4c-b03f-290b9efa1255\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") "
Jul 14 11:30:28.298653 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.298549    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9hjx\" (UniqueName: \"kubernetes.io/projected/42534c61-e8af-4a4c-b03f-290b9efa1255-kube-api-access-p9hjx\") pod \"42534c61-e8af-4a4c-b03f-290b9efa1255\" (UID: \"42534c61-e8af-4a4c-b03f-290b9efa1255\") "
Jul 14 11:30:28.300587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.300551    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42534c61-e8af-4a4c-b03f-290b9efa1255-kube-api-access-p9hjx" (OuterVolumeSpecName: "kube-api-access-p9hjx") pod "42534c61-e8af-4a4c-b03f-290b9efa1255" (UID: "42534c61-e8af-4a4c-b03f-290b9efa1255"). InnerVolumeSpecName "kube-api-access-p9hjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:30:28.309632 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.309594    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42534c61-e8af-4a4c-b03f-290b9efa1255-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "42534c61-e8af-4a4c-b03f-290b9efa1255" (UID: "42534c61-e8af-4a4c-b03f-290b9efa1255"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:30:28.400068 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.400027    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42534c61-e8af-4a4c-b03f-290b9efa1255-tls-cert\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:30:28.400068 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.400070    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9hjx\" (UniqueName: \"kubernetes.io/projected/42534c61-e8af-4a4c-b03f-290b9efa1255-kube-api-access-p9hjx\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:30:28.863927 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.863890    2564 generic.go:356] "Generic (PLEG): container finished" podID="42534c61-e8af-4a4c-b03f-290b9efa1255" containerID="e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31" exitCode=0
Jul 14 11:30:28.864343 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.863934    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5"
Jul 14 11:30:28.864343 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.863976    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5" event={"ID":"42534c61-e8af-4a4c-b03f-290b9efa1255","Type":"ContainerDied","Data":"e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31"}
Jul 14 11:30:28.864343 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.864019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bdcd6b6f6-g58x5" event={"ID":"42534c61-e8af-4a4c-b03f-290b9efa1255","Type":"ContainerDied","Data":"21fe1ddd948331c67bf89d9e4337b75fc12e92a4d03c6a4d39d5e290fe91311a"}
Jul 14 11:30:28.864343 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.864040    2564 scope.go:117] "RemoveContainer" containerID="e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31"
Jul 14 11:30:28.872563 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.872538    2564 scope.go:117] "RemoveContainer" containerID="e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31"
Jul 14 11:30:28.872852 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:30:28.872819    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31\": container with ID starting with e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31 not found: ID does not exist" containerID="e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31"
Jul 14 11:30:28.872957 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.872851    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31"} err="failed to get container status \"e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31\": rpc error: code = NotFound desc = could not find container \"e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31\": container with ID starting with e974986e2596952dea2bc0c23f4fb5b1917ec90c1502b8522752738f9c922a31 not found: ID does not exist"
Jul 14 11:30:28.884480 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.884453    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bdcd6b6f6-g58x5"]
Jul 14 11:30:28.890954 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:28.890927    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bdcd6b6f6-g58x5"]
Jul 14 11:30:30.768906 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:30.768869    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42534c61-e8af-4a4c-b03f-290b9efa1255" path="/var/lib/kubelet/pods/42534c61-e8af-4a4c-b03f-290b9efa1255/volumes"
Jul 14 11:30:54.170136 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:30:54.170096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:31:06.283686 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:31:06.283651    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:31:16.581250 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:31:16.581211    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:31:25.270073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:31:25.270037    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:31:35.879144 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:31:35.879099    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:32:46.279234 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:32:46.279191    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:32:55.810878 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:32:55.810839    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:33:07.174941 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:33:07.174902    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:33:18.671504 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:33:18.671414    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:33:26.642515 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:33:26.642483    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:33:26.643847 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:33:26.643826    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:33:48.273048 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:33:48.273011    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:33:58.077924 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:33:58.077888    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:34:20.479718 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:34:20.479662    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:34:32.265450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:34:32.265413    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:34:54.980775 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:34:54.980682    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:35:06.873203 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:35:06.873163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:35:21.978384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:35:21.978345    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:35:32.974590 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:35:32.974543    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:35:42.370591 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:35:42.370546    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:35:51.568499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:35:51.568458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:36:00.866688 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:36:00.866644    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:37:50.071009 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:37:50.070966    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:38:00.472323 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:00.472285    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:38:09.687866 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:09.687833    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:38:21.994962 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:21.994919    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:38:26.668640 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:26.668610    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:38:26.671011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:26.670993    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:38:31.765324 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:31.765282    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:38:43.366203 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:43.366165    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:38:52.462813 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:38:52.462774    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:39:04.398491 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:39:04.398458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:39:13.884999 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:39:13.884959    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:39:22.770479 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:39:22.770405    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:39:34.078800 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:39:34.078764    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:39:55.276642 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:39:55.276610    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:40:04.296593 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:40:04.296550    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:40:13.375343 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:40:13.375299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:40:25.572691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:40:25.572654    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:40:43.382986 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:40:43.382952    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:40:51.467762 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:40:51.467671    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:41:00.367093 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:41:00.367061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:41:11.073580 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:41:11.073541    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:41:20.286672 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:41:20.286615    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:41:29.366294 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:41:29.366262    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:41:43.682057 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:41:43.682016    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:41:52.774390 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:41:52.774349    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:42:02.974848 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:42:02.974809    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:42:15.169806 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:42:15.169767    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:42:24.372436 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:42:24.372356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:42:31.171042 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:42:31.170989    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:42:49.662713 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:42:49.662669    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:43:00.680792 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:43:00.680753    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:43:14.872688 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:43:14.872650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:43:26.694340 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:43:26.694308    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:43:26.697835 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:43:26.697814    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:43:34.970729 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:43:34.970679    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:44:45.966688 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:44:45.966657    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:45:00.132141 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.132086    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw"]
Jul 14 11:45:00.132607 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.132590    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42534c61-e8af-4a4c-b03f-290b9efa1255" containerName="authorino"
Jul 14 11:45:00.132658 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.132611    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="42534c61-e8af-4a4c-b03f-290b9efa1255" containerName="authorino"
Jul 14 11:45:00.132748 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.132735    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="42534c61-e8af-4a4c-b03f-290b9efa1255" containerName="authorino"
Jul 14 11:45:00.136031 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.136012    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"]
Jul 14 11:45:00.136150 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.136133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw"
Jul 14 11:45:00.138403 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.138384    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:45:00.139144 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.139128    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"
Jul 14 11:45:00.142900 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.142877    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"]
Jul 14 11:45:00.145918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.145898    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw"]
Jul 14 11:45:00.149902 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.149886    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw"
Jul 14 11:45:00.157141 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.157125    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"
Jul 14 11:45:00.295740 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.295713    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"]
Jul 14 11:45:00.298432 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:45:00.298403    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e3abcb_63c0_4adb_a0d9_5b68d7a6c5ef.slice/crio-0e99473c72cbbfd53f7d3271a5eb073e538f2476193b7c26abbefd974028c1b4 WatchSource:0}: Error finding container 0e99473c72cbbfd53f7d3271a5eb073e538f2476193b7c26abbefd974028c1b4: Status 404 returned error can't find the container with id 0e99473c72cbbfd53f7d3271a5eb073e538f2476193b7c26abbefd974028c1b4
Jul 14 11:45:00.302116 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.301011    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 11:45:00.528897 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.528867    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw"]
Jul 14 11:45:00.532391 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:45:00.532360    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b222f5d_91b7_455a_8eb1_422d69461b12.slice/crio-4144b38d2da6f2badfab76a32095f601286b01f0dc0dacf8adfb24fb19bcbb01 WatchSource:0}: Error finding container 4144b38d2da6f2badfab76a32095f601286b01f0dc0dacf8adfb24fb19bcbb01: Status 404 returned error can't find the container with id 4144b38d2da6f2badfab76a32095f601286b01f0dc0dacf8adfb24fb19bcbb01
Jul 14 11:45:00.741722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.741667    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" event={"ID":"f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef","Type":"ContainerStarted","Data":"0e99473c72cbbfd53f7d3271a5eb073e538f2476193b7c26abbefd974028c1b4"}
Jul 14 11:45:00.742830 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:00.742802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw" event={"ID":"7b222f5d-91b7-455a-8eb1-422d69461b12","Type":"ContainerStarted","Data":"4144b38d2da6f2badfab76a32095f601286b01f0dc0dacf8adfb24fb19bcbb01"}
Jul 14 11:45:01.868105 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:01.868066    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:45:02.754444 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:02.754405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw" event={"ID":"7b222f5d-91b7-455a-8eb1-422d69461b12","Type":"ContainerStarted","Data":"efb194c298b40e25cbc1192083a35e168cbd5e7551c8d57d477eac24f48f95a4"}
Jul 14 11:45:02.755857 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:02.755828    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" event={"ID":"f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef","Type":"ContainerStarted","Data":"70e2383ab9eeb5429e98224865e9e8878de49b23efcac6155957507817764993"}
Jul 14 11:45:02.770432 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:02.770379    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw" podStartSLOduration=0.987223608 podStartE2EDuration="2.770363831s" totalImagesPullingTime="1.783140223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:45:02.768899097 +0000 UTC m=+1596.569454837" watchObservedRunningTime="2026-07-14 11:45:02.770363831 +0000 UTC m=+1596.570919570"
Jul 14 11:45:02.784902 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:02.784856    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" podStartSLOduration=0.772957325 podStartE2EDuration="2.784842798s" totalImagesPullingTime="2.011885473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:45:02.783072691 +0000 UTC m=+1596.583628432" watchObservedRunningTime="2026-07-14 11:45:02.784842798 +0000 UTC m=+1596.585398534"
Jul 14 11:45:03.767505 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:03.767469    2564 generic.go:356] "Generic (PLEG): container finished" podID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerID="70e2383ab9eeb5429e98224865e9e8878de49b23efcac6155957507817764993" exitCode=7
Jul 14 11:45:03.767953 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:03.767560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" event={"ID":"f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef","Type":"ContainerDied","Data":"70e2383ab9eeb5429e98224865e9e8878de49b23efcac6155957507817764993"}
Jul 14 11:45:03.767953 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:03.767672    2564 scope.go:117] "RemoveContainer" containerID="70e2383ab9eeb5429e98224865e9e8878de49b23efcac6155957507817764993"
Jul 14 11:45:03.769186 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:03.769166    2564 generic.go:356] "Generic (PLEG): container finished" podID="7b222f5d-91b7-455a-8eb1-422d69461b12" containerID="efb194c298b40e25cbc1192083a35e168cbd5e7551c8d57d477eac24f48f95a4" exitCode=0
Jul 14 11:45:03.769287 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:03.769199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw" event={"ID":"7b222f5d-91b7-455a-8eb1-422d69461b12","Type":"ContainerDied","Data":"efb194c298b40e25cbc1192083a35e168cbd5e7551c8d57d477eac24f48f95a4"}
Jul 14 11:45:04.101866 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.101833    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"]
Jul 14 11:45:04.105122 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.105107    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"
Jul 14 11:45:04.107236 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.107213    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:45:04.113635 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.113608    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"]
Jul 14 11:45:04.274275 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.274244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2031527c-5718-4bf6-8131-48cd4a870657-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8\" (UID: \"2031527c-5718-4bf6-8131-48cd4a870657\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"
Jul 14 11:45:04.375129 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.375106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2031527c-5718-4bf6-8131-48cd4a870657-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8\" (UID: \"2031527c-5718-4bf6-8131-48cd4a870657\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"
Jul 14 11:45:04.375643 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.375624    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2031527c-5718-4bf6-8131-48cd4a870657-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8\" (UID: \"2031527c-5718-4bf6-8131-48cd4a870657\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"
Jul 14 11:45:04.416181 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.416151    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"
Jul 14 11:45:04.545329 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.545304    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"]
Jul 14 11:45:04.547181 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:45:04.547144    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2031527c_5718_4bf6_8131_48cd4a870657.slice/crio-35c93647d664631aee73987c0d03887019f1aaccbd3534ea2e183f9845bd8777 WatchSource:0}: Error finding container 35c93647d664631aee73987c0d03887019f1aaccbd3534ea2e183f9845bd8777: Status 404 returned error can't find the container with id 35c93647d664631aee73987c0d03887019f1aaccbd3534ea2e183f9845bd8777
Jul 14 11:45:04.774891 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.774847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8" event={"ID":"2031527c-5718-4bf6-8131-48cd4a870657","Type":"ContainerStarted","Data":"35c93647d664631aee73987c0d03887019f1aaccbd3534ea2e183f9845bd8777"}
Jul 14 11:45:04.776619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.776588    2564 generic.go:356] "Generic (PLEG): container finished" podID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerID="deec5db33d2ef8ccbf8d705fa94040d76da6632629a763256b8dee4017d5403f" exitCode=7
Jul 14 11:45:04.776758 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.776677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" event={"ID":"f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef","Type":"ContainerDied","Data":"deec5db33d2ef8ccbf8d705fa94040d76da6632629a763256b8dee4017d5403f"}
Jul 14 11:45:04.776758 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.776745    2564 scope.go:117] "RemoveContainer" containerID="70e2383ab9eeb5429e98224865e9e8878de49b23efcac6155957507817764993"
Jul 14 11:45:04.776840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.776830    2564 scope.go:117] "RemoveContainer" containerID="deec5db33d2ef8ccbf8d705fa94040d76da6632629a763256b8dee4017d5403f"
Jul 14 11:45:04.777416 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:45:04.777066    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x_odh-ai-gateway-infra(f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef"
Jul 14 11:45:04.904921 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:04.904900    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw"
Jul 14 11:45:05.782806 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:05.782778    2564 scope.go:117] "RemoveContainer" containerID="deec5db33d2ef8ccbf8d705fa94040d76da6632629a763256b8dee4017d5403f"
Jul 14 11:45:05.783242 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:45:05.782989    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x_odh-ai-gateway-infra(f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef"
Jul 14 11:45:05.783848 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:05.783827    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw"
Jul 14 11:45:05.783928 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:05.783830    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733825-r28lw" event={"ID":"7b222f5d-91b7-455a-8eb1-422d69461b12","Type":"ContainerDied","Data":"4144b38d2da6f2badfab76a32095f601286b01f0dc0dacf8adfb24fb19bcbb01"}
Jul 14 11:45:05.783996 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:05.783957    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4144b38d2da6f2badfab76a32095f601286b01f0dc0dacf8adfb24fb19bcbb01"
Jul 14 11:45:05.785188 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:05.785165    2564 generic.go:356] "Generic (PLEG): container finished" podID="2031527c-5718-4bf6-8131-48cd4a870657" containerID="fe3a1599f5c138c885ead75c8a3b6df49655ede7ce147da252c62187aa9271a0" exitCode=7
Jul 14 11:45:05.785295 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:05.785187    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8" event={"ID":"2031527c-5718-4bf6-8131-48cd4a870657","Type":"ContainerDied","Data":"fe3a1599f5c138c885ead75c8a3b6df49655ede7ce147da252c62187aa9271a0"}
Jul 14 11:45:05.785451 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:05.785435    2564 scope.go:117] "RemoveContainer" containerID="fe3a1599f5c138c885ead75c8a3b6df49655ede7ce147da252c62187aa9271a0"
Jul 14 11:45:06.791713 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:06.791660    2564 generic.go:356] "Generic (PLEG): container finished" podID="2031527c-5718-4bf6-8131-48cd4a870657" containerID="cd8a0878136261e8522177d7a549d171675226c47a7379f809519554461ff97c" exitCode=7
Jul 14 11:45:06.792154 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:06.791731    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8" event={"ID":"2031527c-5718-4bf6-8131-48cd4a870657","Type":"ContainerDied","Data":"cd8a0878136261e8522177d7a549d171675226c47a7379f809519554461ff97c"}
Jul 14 11:45:06.792154 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:06.791799    2564 scope.go:117] "RemoveContainer" containerID="fe3a1599f5c138c885ead75c8a3b6df49655ede7ce147da252c62187aa9271a0"
Jul 14 11:45:06.792154 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:06.791981    2564 scope.go:117] "RemoveContainer" containerID="cd8a0878136261e8522177d7a549d171675226c47a7379f809519554461ff97c"
Jul 14 11:45:06.792292 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:45:06.792220    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8_odh-ai-gateway-infra(2031527c-5718-4bf6-8131-48cd4a870657)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8" podUID="2031527c-5718-4bf6-8131-48cd4a870657"
Jul 14 11:45:07.798171 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:07.798133    2564 scope.go:117] "RemoveContainer" containerID="cd8a0878136261e8522177d7a549d171675226c47a7379f809519554461ff97c"
Jul 14 11:45:07.798714 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:45:07.798323    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8_odh-ai-gateway-infra(2031527c-5718-4bf6-8131-48cd4a870657)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8" podUID="2031527c-5718-4bf6-8131-48cd4a870657"
Jul 14 11:45:20.764251 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:20.764177    2564 scope.go:117] "RemoveContainer" containerID="deec5db33d2ef8ccbf8d705fa94040d76da6632629a763256b8dee4017d5403f"
Jul 14 11:45:21.861036 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:21.861002    2564 generic.go:356] "Generic (PLEG): container finished" podID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerID="9ffa6f8b8319a09f3c886749d4f1fb92f3f8e669604818217ef341b7e2f9e05d" exitCode=0
Jul 14 11:45:21.861414 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:21.861077    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" event={"ID":"f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef","Type":"ContainerDied","Data":"9ffa6f8b8319a09f3c886749d4f1fb92f3f8e669604818217ef341b7e2f9e05d"}
Jul 14 11:45:21.861414 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:21.861161    2564 scope.go:117] "RemoveContainer" containerID="deec5db33d2ef8ccbf8d705fa94040d76da6632629a763256b8dee4017d5403f"
Jul 14 11:45:22.764122 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:22.764090    2564 scope.go:117] "RemoveContainer" containerID="cd8a0878136261e8522177d7a549d171675226c47a7379f809519554461ff97c"
Jul 14 11:45:22.893950 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:22.893910    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"]
Jul 14 11:45:23.004521 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.004500    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"
Jul 14 11:45:23.874146 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.874112    2564 generic.go:356] "Generic (PLEG): container finished" podID="2031527c-5718-4bf6-8131-48cd4a870657" containerID="700f9d74943b4c66fe6b2489c65e07ef2218cf4e8298b43612e7c2b432b28845" exitCode=0
Jul 14 11:45:23.874334 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.874181    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8" event={"ID":"2031527c-5718-4bf6-8131-48cd4a870657","Type":"ContainerDied","Data":"700f9d74943b4c66fe6b2489c65e07ef2218cf4e8298b43612e7c2b432b28845"}
Jul 14 11:45:23.874334 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.874240    2564 scope.go:117] "RemoveContainer" containerID="cd8a0878136261e8522177d7a549d171675226c47a7379f809519554461ff97c"
Jul 14 11:45:23.875775 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.875753    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x" event={"ID":"f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef","Type":"ContainerDied","Data":"0e99473c72cbbfd53f7d3271a5eb073e538f2476193b7c26abbefd974028c1b4"}
Jul 14 11:45:23.875907 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.875791    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e99473c72cbbfd53f7d3271a5eb073e538f2476193b7c26abbefd974028c1b4"
Jul 14 11:45:23.875907 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.875794    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"
Jul 14 11:45:23.929605 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.929573    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"]
Jul 14 11:45:23.936047 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:23.936023    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-f22ceb3a-29733825-q2l9x"]
Jul 14 11:45:24.769008 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:24.768969    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" path="/var/lib/kubelet/pods/f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef/volumes"
Jul 14 11:45:24.917755 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:24.917725    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"]
Jul 14 11:45:25.011240 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.011218    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"
Jul 14 11:45:25.046403 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.046330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2031527c-5718-4bf6-8131-48cd4a870657-maas-api-service-ca\") pod \"2031527c-5718-4bf6-8131-48cd4a870657\" (UID: \"2031527c-5718-4bf6-8131-48cd4a870657\") "
Jul 14 11:45:25.046641 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.046618    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2031527c-5718-4bf6-8131-48cd4a870657-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2031527c-5718-4bf6-8131-48cd4a870657" (UID: "2031527c-5718-4bf6-8131-48cd4a870657"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:45:25.147549 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.147509    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2031527c-5718-4bf6-8131-48cd4a870657-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:45:25.887279 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.887239    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8" event={"ID":"2031527c-5718-4bf6-8131-48cd4a870657","Type":"ContainerDied","Data":"35c93647d664631aee73987c0d03887019f1aaccbd3534ea2e183f9845bd8777"}
Jul 14 11:45:25.887279 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.887268    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"
Jul 14 11:45:25.887542 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.887291    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35c93647d664631aee73987c0d03887019f1aaccbd3534ea2e183f9845bd8777"
Jul 14 11:45:25.921434 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.921408    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"]
Jul 14 11:45:25.927558 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:25.927536    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-pkqk8"]
Jul 14 11:45:26.770440 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:45:26.770397    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2031527c-5718-4bf6-8131-48cd4a870657" path="/var/lib/kubelet/pods/2031527c-5718-4bf6-8131-48cd4a870657/volumes"
Jul 14 11:46:04.165571 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165538    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"]
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165874    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165885    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165895    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165903    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165920    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b222f5d-91b7-455a-8eb1-422d69461b12" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165925    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b222f5d-91b7-455a-8eb1-422d69461b12" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165932    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165937    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165944    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165949    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.165995    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b222f5d-91b7-455a-8eb1-422d69461b12" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.166003    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.166010    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.166016    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:04.166053 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.166023    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:04.168017 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.168000    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"
Jul 14 11:46:04.170224 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.170200    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:46:04.170364 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.170306    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:46:04.176160 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.176136    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"]
Jul 14 11:46:04.279177 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.279137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8976e84c-056f-47b0-885b-c7be26f436b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf\" (UID: \"8976e84c-056f-47b0-885b-c7be26f436b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"
Jul 14 11:46:04.380283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.380242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8976e84c-056f-47b0-885b-c7be26f436b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf\" (UID: \"8976e84c-056f-47b0-885b-c7be26f436b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"
Jul 14 11:46:04.380906 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.380886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8976e84c-056f-47b0-885b-c7be26f436b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf\" (UID: \"8976e84c-056f-47b0-885b-c7be26f436b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"
Jul 14 11:46:04.479098 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.479010    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"
Jul 14 11:46:04.609274 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:04.609200    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"]
Jul 14 11:46:04.611679 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:46:04.611652    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8976e84c_056f_47b0_885b_c7be26f436b5.slice/crio-13efbd2de0042ba0791a8fac39ae5c6ff2913f6c583be5937d2a91cfbe675301 WatchSource:0}: Error finding container 13efbd2de0042ba0791a8fac39ae5c6ff2913f6c583be5937d2a91cfbe675301: Status 404 returned error can't find the container with id 13efbd2de0042ba0791a8fac39ae5c6ff2913f6c583be5937d2a91cfbe675301
Jul 14 11:46:05.061554 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:05.061516    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf" event={"ID":"8976e84c-056f-47b0-885b-c7be26f436b5","Type":"ContainerStarted","Data":"4f54b53b7d8e8e220153cc16d1178d56c14dc810d161fc9dd4f50191fa2c2006"}
Jul 14 11:46:05.061758 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:05.061574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf" event={"ID":"8976e84c-056f-47b0-885b-c7be26f436b5","Type":"ContainerStarted","Data":"13efbd2de0042ba0791a8fac39ae5c6ff2913f6c583be5937d2a91cfbe675301"}
Jul 14 11:46:05.078578 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:05.078520    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf" podStartSLOduration=1.078505154 podStartE2EDuration="1.078505154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:46:05.076912262 +0000 UTC m=+1658.877468018" watchObservedRunningTime="2026-07-14 11:46:05.078505154 +0000 UTC m=+1658.879060893"
Jul 14 11:46:06.067231 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:06.067196    2564 generic.go:356] "Generic (PLEG): container finished" podID="8976e84c-056f-47b0-885b-c7be26f436b5" containerID="4f54b53b7d8e8e220153cc16d1178d56c14dc810d161fc9dd4f50191fa2c2006" exitCode=0
Jul 14 11:46:06.067595 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:06.067256    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf" event={"ID":"8976e84c-056f-47b0-885b-c7be26f436b5","Type":"ContainerDied","Data":"4f54b53b7d8e8e220153cc16d1178d56c14dc810d161fc9dd4f50191fa2c2006"}
Jul 14 11:46:07.205080 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:07.205051    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"
Jul 14 11:46:07.224305 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:07.224277    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:46:07.304784 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:07.304746    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8976e84c-056f-47b0-885b-c7be26f436b5-maas-api-service-ca\") pod \"8976e84c-056f-47b0-885b-c7be26f436b5\" (UID: \"8976e84c-056f-47b0-885b-c7be26f436b5\") "
Jul 14 11:46:07.305113 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:07.305087    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8976e84c-056f-47b0-885b-c7be26f436b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8976e84c-056f-47b0-885b-c7be26f436b5" (UID: "8976e84c-056f-47b0-885b-c7be26f436b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:46:07.406264 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:07.406184    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8976e84c-056f-47b0-885b-c7be26f436b5-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:46:08.076293 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:08.076262    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf"
Jul 14 11:46:08.076293 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:08.076288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf" event={"ID":"8976e84c-056f-47b0-885b-c7be26f436b5","Type":"ContainerDied","Data":"13efbd2de0042ba0791a8fac39ae5c6ff2913f6c583be5937d2a91cfbe675301"}
Jul 14 11:46:08.076550 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:08.076340    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13efbd2de0042ba0791a8fac39ae5c6ff2913f6c583be5937d2a91cfbe675301"
Jul 14 11:46:20.681056 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:20.681015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:46:22.869123 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869082    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"]
Jul 14 11:46:22.869592 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869552    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:22.869592 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869570    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:22.869592 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869583    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:22.869592 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869592    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:22.869840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869624    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8976e84c-056f-47b0-885b-c7be26f436b5" containerName="revoke-keys"
Jul 14 11:46:22.869840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869634    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8976e84c-056f-47b0-885b-c7be26f436b5" containerName="revoke-keys"
Jul 14 11:46:22.869840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869731    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8976e84c-056f-47b0-885b-c7be26f436b5" containerName="revoke-keys"
Jul 14 11:46:22.869840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869745    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2031527c-5718-4bf6-8131-48cd4a870657" containerName="revoke-keys"
Jul 14 11:46:22.869840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.869758    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e3abcb-63c0-4adb-a0d9-5b68d7a6c5ef" containerName="cleanup"
Jul 14 11:46:22.871849 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.871828    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"
Jul 14 11:46:22.874052 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.874030    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:46:22.874154 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.874066    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:46:22.879831 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.879811    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"]
Jul 14 11:46:22.942874 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:22.942842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/237e874a-aa0a-4e09-af07-da20b7f17a8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4b103af8-kchv9\" (UID: \"237e874a-aa0a-4e09-af07-da20b7f17a8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"
Jul 14 11:46:23.044292 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:23.044247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/237e874a-aa0a-4e09-af07-da20b7f17a8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4b103af8-kchv9\" (UID: \"237e874a-aa0a-4e09-af07-da20b7f17a8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"
Jul 14 11:46:23.044941 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:23.044924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/237e874a-aa0a-4e09-af07-da20b7f17a8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4b103af8-kchv9\" (UID: \"237e874a-aa0a-4e09-af07-da20b7f17a8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"
Jul 14 11:46:23.182664 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:23.182572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"
Jul 14 11:46:23.306514 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:23.306493    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"]
Jul 14 11:46:23.308499 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:46:23.308472    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237e874a_aa0a_4e09_af07_da20b7f17a8f.slice/crio-316cafc516264402f2d2761c46a921e3211caf769246e0e1dcce5d4eb2149b39 WatchSource:0}: Error finding container 316cafc516264402f2d2761c46a921e3211caf769246e0e1dcce5d4eb2149b39: Status 404 returned error can't find the container with id 316cafc516264402f2d2761c46a921e3211caf769246e0e1dcce5d4eb2149b39
Jul 14 11:46:24.147846 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:24.147755    2564 generic.go:356] "Generic (PLEG): container finished" podID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerID="92d12980a4a96fdc5f4578a135f210ef2e7a840390d57ee9ca7ee79a7ff8ba85" exitCode=7
Jul 14 11:46:24.148285 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:24.147840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" event={"ID":"237e874a-aa0a-4e09-af07-da20b7f17a8f","Type":"ContainerDied","Data":"92d12980a4a96fdc5f4578a135f210ef2e7a840390d57ee9ca7ee79a7ff8ba85"}
Jul 14 11:46:24.148285 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:24.147885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" event={"ID":"237e874a-aa0a-4e09-af07-da20b7f17a8f","Type":"ContainerStarted","Data":"316cafc516264402f2d2761c46a921e3211caf769246e0e1dcce5d4eb2149b39"}
Jul 14 11:46:24.148285 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:24.148025    2564 scope.go:117] "RemoveContainer" containerID="92d12980a4a96fdc5f4578a135f210ef2e7a840390d57ee9ca7ee79a7ff8ba85"
Jul 14 11:46:25.154307 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:25.154267    2564 generic.go:356] "Generic (PLEG): container finished" podID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerID="dcd30193f8d84120f232832a825e790b4b6f9bf80aa0d8361cfa616c1d86c841" exitCode=7
Jul 14 11:46:25.154722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:25.154354    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" event={"ID":"237e874a-aa0a-4e09-af07-da20b7f17a8f","Type":"ContainerDied","Data":"dcd30193f8d84120f232832a825e790b4b6f9bf80aa0d8361cfa616c1d86c841"}
Jul 14 11:46:25.154722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:25.154417    2564 scope.go:117] "RemoveContainer" containerID="92d12980a4a96fdc5f4578a135f210ef2e7a840390d57ee9ca7ee79a7ff8ba85"
Jul 14 11:46:25.154722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:25.154651    2564 scope.go:117] "RemoveContainer" containerID="dcd30193f8d84120f232832a825e790b4b6f9bf80aa0d8361cfa616c1d86c841"
Jul 14 11:46:25.154896 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:46:25.154877    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4b103af8-kchv9_odh-ai-gateway-infra(237e874a-aa0a-4e09-af07-da20b7f17a8f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f"
Jul 14 11:46:26.160303 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:26.160278    2564 scope.go:117] "RemoveContainer" containerID="dcd30193f8d84120f232832a825e790b4b6f9bf80aa0d8361cfa616c1d86c841"
Jul 14 11:46:26.160689 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:46:26.160480    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4b103af8-kchv9_odh-ai-gateway-infra(237e874a-aa0a-4e09-af07-da20b7f17a8f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f"
Jul 14 11:46:36.766992 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:36.766961    2564 scope.go:117] "RemoveContainer" containerID="dcd30193f8d84120f232832a825e790b4b6f9bf80aa0d8361cfa616c1d86c841"
Jul 14 11:46:37.211939 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:37.211899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" event={"ID":"237e874a-aa0a-4e09-af07-da20b7f17a8f","Type":"ContainerStarted","Data":"c38c42fa8ddb262be9b47c35b17bf70a60fc45ab7e86452a5b2818f41a58c8d9"}
Jul 14 11:46:37.228274 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:37.228215    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" podStartSLOduration=15.228200884 podStartE2EDuration="15.228200884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:46:37.227440826 +0000 UTC m=+1691.027996577" watchObservedRunningTime="2026-07-14 11:46:37.228200884 +0000 UTC m=+1691.028756624"
Jul 14 11:46:38.217980 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:38.217945    2564 generic.go:356] "Generic (PLEG): container finished" podID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerID="c38c42fa8ddb262be9b47c35b17bf70a60fc45ab7e86452a5b2818f41a58c8d9" exitCode=0
Jul 14 11:46:38.218420 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:38.218017    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" event={"ID":"237e874a-aa0a-4e09-af07-da20b7f17a8f","Type":"ContainerDied","Data":"c38c42fa8ddb262be9b47c35b17bf70a60fc45ab7e86452a5b2818f41a58c8d9"}
Jul 14 11:46:38.218420 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:38.218097    2564 scope.go:117] "RemoveContainer" containerID="dcd30193f8d84120f232832a825e790b4b6f9bf80aa0d8361cfa616c1d86c841"
Jul 14 11:46:38.244210 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:38.244178    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"]
Jul 14 11:46:39.354905 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:39.354881    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"
Jul 14 11:46:39.487100 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:39.487003    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/237e874a-aa0a-4e09-af07-da20b7f17a8f-maas-api-service-ca\") pod \"237e874a-aa0a-4e09-af07-da20b7f17a8f\" (UID: \"237e874a-aa0a-4e09-af07-da20b7f17a8f\") "
Jul 14 11:46:39.487367 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:39.487342    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/237e874a-aa0a-4e09-af07-da20b7f17a8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "237e874a-aa0a-4e09-af07-da20b7f17a8f" (UID: "237e874a-aa0a-4e09-af07-da20b7f17a8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:46:39.588340 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:39.588287    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/237e874a-aa0a-4e09-af07-da20b7f17a8f-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:46:40.229563 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:40.229532    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"
Jul 14 11:46:40.229781 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:40.229564    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9" event={"ID":"237e874a-aa0a-4e09-af07-da20b7f17a8f","Type":"ContainerDied","Data":"316cafc516264402f2d2761c46a921e3211caf769246e0e1dcce5d4eb2149b39"}
Jul 14 11:46:40.229781 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:40.229609    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="316cafc516264402f2d2761c46a921e3211caf769246e0e1dcce5d4eb2149b39"
Jul 14 11:46:40.256537 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:40.256510    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"]
Jul 14 11:46:40.260621 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:40.260596    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-kchv9"]
Jul 14 11:46:40.769286 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:46:40.769253    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" path="/var/lib/kubelet/pods/237e874a-aa0a-4e09-af07-da20b7f17a8f/volumes"
Jul 14 11:47:12.919021 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.918940    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"]
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919291    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919305    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919317    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919323    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919331    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919336    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919401    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.919465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.919413    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:12.921452 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.921436    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"
Jul 14 11:47:12.923966 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.923945    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:47:12.924225 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.924210    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:47:12.930324 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.930299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"]
Jul 14 11:47:12.948332 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:12.948302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b32a5ce-ce3a-4801-a7ee-932b0df3664f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2\" (UID: \"2b32a5ce-ce3a-4801-a7ee-932b0df3664f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"
Jul 14 11:47:13.049074 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:13.049034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b32a5ce-ce3a-4801-a7ee-932b0df3664f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2\" (UID: \"2b32a5ce-ce3a-4801-a7ee-932b0df3664f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"
Jul 14 11:47:13.049640 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:13.049617    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b32a5ce-ce3a-4801-a7ee-932b0df3664f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2\" (UID: \"2b32a5ce-ce3a-4801-a7ee-932b0df3664f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"
Jul 14 11:47:13.233918 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:13.233827    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"
Jul 14 11:47:13.363174 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:13.363149    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"]
Jul 14 11:47:13.365348 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:47:13.365319    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b32a5ce_ce3a_4801_a7ee_932b0df3664f.slice/crio-877657ea665a0d855c2e8f056bd5cd81c30cbcc46b10452613054e9136adb81b WatchSource:0}: Error finding container 877657ea665a0d855c2e8f056bd5cd81c30cbcc46b10452613054e9136adb81b: Status 404 returned error can't find the container with id 877657ea665a0d855c2e8f056bd5cd81c30cbcc46b10452613054e9136adb81b
Jul 14 11:47:13.384538 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:13.384512    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2" event={"ID":"2b32a5ce-ce3a-4801-a7ee-932b0df3664f","Type":"ContainerStarted","Data":"877657ea665a0d855c2e8f056bd5cd81c30cbcc46b10452613054e9136adb81b"}
Jul 14 11:47:14.390679 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:14.390640    2564 generic.go:356] "Generic (PLEG): container finished" podID="2b32a5ce-ce3a-4801-a7ee-932b0df3664f" containerID="632c83c630e98ac3b53d9bada47cafa86d3e5bdaf0f4eb6988286948f4d22f6b" exitCode=0
Jul 14 11:47:14.391069 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:14.390711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2" event={"ID":"2b32a5ce-ce3a-4801-a7ee-932b0df3664f","Type":"ContainerDied","Data":"632c83c630e98ac3b53d9bada47cafa86d3e5bdaf0f4eb6988286948f4d22f6b"}
Jul 14 11:47:15.519370 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:15.519344    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"
Jul 14 11:47:15.572476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:15.572445    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b32a5ce-ce3a-4801-a7ee-932b0df3664f-maas-api-service-ca\") pod \"2b32a5ce-ce3a-4801-a7ee-932b0df3664f\" (UID: \"2b32a5ce-ce3a-4801-a7ee-932b0df3664f\") "
Jul 14 11:47:15.572804 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:15.572780    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b32a5ce-ce3a-4801-a7ee-932b0df3664f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b32a5ce-ce3a-4801-a7ee-932b0df3664f" (UID: "2b32a5ce-ce3a-4801-a7ee-932b0df3664f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:47:15.673727 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:15.673634    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b32a5ce-ce3a-4801-a7ee-932b0df3664f-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:47:16.401599 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:16.401566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2" event={"ID":"2b32a5ce-ce3a-4801-a7ee-932b0df3664f","Type":"ContainerDied","Data":"877657ea665a0d855c2e8f056bd5cd81c30cbcc46b10452613054e9136adb81b"}
Jul 14 11:47:16.401794 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:16.401618    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="877657ea665a0d855c2e8f056bd5cd81c30cbcc46b10452613054e9136adb81b"
Jul 14 11:47:16.401794 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:16.401585    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2"
Jul 14 11:47:26.361432 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:26.361396    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:47:42.875925 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:42.875891    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:47:44.847896 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.847865    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"]
Jul 14 11:47:44.848243 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.848199    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b32a5ce-ce3a-4801-a7ee-932b0df3664f" containerName="revoke-keys"
Jul 14 11:47:44.848243 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.848209    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b32a5ce-ce3a-4801-a7ee-932b0df3664f" containerName="revoke-keys"
Jul 14 11:47:44.848326 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.848270    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="237e874a-aa0a-4e09-af07-da20b7f17a8f" containerName="revoke-keys"
Jul 14 11:47:44.848326 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.848289    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b32a5ce-ce3a-4801-a7ee-932b0df3664f" containerName="revoke-keys"
Jul 14 11:47:44.850432 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.850416    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"
Jul 14 11:47:44.852765 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.852742    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:47:44.852871 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.852800    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:47:44.863166 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.863142    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"]
Jul 14 11:47:44.930751 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:44.930714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11cca3c9-ddc6-4e64-80e7-9b046bc8e2af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h\" (UID: \"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"
Jul 14 11:47:45.032006 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:45.031973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11cca3c9-ddc6-4e64-80e7-9b046bc8e2af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h\" (UID: \"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"
Jul 14 11:47:45.032578 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:45.032559    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11cca3c9-ddc6-4e64-80e7-9b046bc8e2af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h\" (UID: \"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"
Jul 14 11:47:45.160118 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:45.160043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"
Jul 14 11:47:45.295801 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:45.295759    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"]
Jul 14 11:47:45.299107 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:47:45.299075    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cca3c9_ddc6_4e64_80e7_9b046bc8e2af.slice/crio-2b556c3fc2112622721c8ad3de747ca527fbafb7ff7d543e4735c9b4c17b3f5e WatchSource:0}: Error finding container 2b556c3fc2112622721c8ad3de747ca527fbafb7ff7d543e4735c9b4c17b3f5e: Status 404 returned error can't find the container with id 2b556c3fc2112622721c8ad3de747ca527fbafb7ff7d543e4735c9b4c17b3f5e
Jul 14 11:47:45.532724 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:45.532674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h" event={"ID":"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af","Type":"ContainerStarted","Data":"2b556c3fc2112622721c8ad3de747ca527fbafb7ff7d543e4735c9b4c17b3f5e"}
Jul 14 11:47:46.538900 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:46.538871    2564 generic.go:356] "Generic (PLEG): container finished" podID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerID="e01445a85e557819180921ca6ebd8f91230e8da3f495c0a46a88f0bd2e0c98a0" exitCode=7
Jul 14 11:47:46.539281 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:46.538964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h" event={"ID":"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af","Type":"ContainerDied","Data":"e01445a85e557819180921ca6ebd8f91230e8da3f495c0a46a88f0bd2e0c98a0"}
Jul 14 11:47:46.539281 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:46.539136    2564 scope.go:117] "RemoveContainer" containerID="e01445a85e557819180921ca6ebd8f91230e8da3f495c0a46a88f0bd2e0c98a0"
Jul 14 11:47:47.546283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:47.546254    2564 generic.go:356] "Generic (PLEG): container finished" podID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerID="8031cc6ee2fd89a128b2230a8940ac2a8d1a1fbe261485c32ef70969410c27c0" exitCode=7
Jul 14 11:47:47.546687 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:47.546338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h" event={"ID":"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af","Type":"ContainerDied","Data":"8031cc6ee2fd89a128b2230a8940ac2a8d1a1fbe261485c32ef70969410c27c0"}
Jul 14 11:47:47.546687 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:47.546403    2564 scope.go:117] "RemoveContainer" containerID="e01445a85e557819180921ca6ebd8f91230e8da3f495c0a46a88f0bd2e0c98a0"
Jul 14 11:47:47.546687 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:47.546640    2564 scope.go:117] "RemoveContainer" containerID="8031cc6ee2fd89a128b2230a8940ac2a8d1a1fbe261485c32ef70969410c27c0"
Jul 14 11:47:47.546906 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:47:47.546886    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h_odh-ai-gateway-infra(11cca3c9-ddc6-4e64-80e7-9b046bc8e2af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af"
Jul 14 11:47:48.552620 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:47:48.552595    2564 scope.go:117] "RemoveContainer" containerID="8031cc6ee2fd89a128b2230a8940ac2a8d1a1fbe261485c32ef70969410c27c0"
Jul 14 11:47:48.553114 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:47:48.552852    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h_odh-ai-gateway-infra(11cca3c9-ddc6-4e64-80e7-9b046bc8e2af)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af"
Jul 14 11:48:03.763827 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:03.763792    2564 scope.go:117] "RemoveContainer" containerID="8031cc6ee2fd89a128b2230a8940ac2a8d1a1fbe261485c32ef70969410c27c0"
Jul 14 11:48:04.631117 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:04.631085    2564 generic.go:356] "Generic (PLEG): container finished" podID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerID="c7f05a83fd1002edeeefd63ff74173cbd7bfc3bf5bded7281ba10a1ba36907d5" exitCode=0
Jul 14 11:48:04.631262 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:04.631163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h" event={"ID":"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af","Type":"ContainerDied","Data":"c7f05a83fd1002edeeefd63ff74173cbd7bfc3bf5bded7281ba10a1ba36907d5"}
Jul 14 11:48:04.631262 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:04.631231    2564 scope.go:117] "RemoveContainer" containerID="8031cc6ee2fd89a128b2230a8940ac2a8d1a1fbe261485c32ef70969410c27c0"
Jul 14 11:48:05.663245 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:05.663213    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"]
Jul 14 11:48:05.768302 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:05.768266    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"
Jul 14 11:48:05.803283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:05.803246    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11cca3c9-ddc6-4e64-80e7-9b046bc8e2af-maas-api-service-ca\") pod \"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af\" (UID: \"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af\") "
Jul 14 11:48:05.803622 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:05.803597    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11cca3c9-ddc6-4e64-80e7-9b046bc8e2af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" (UID: "11cca3c9-ddc6-4e64-80e7-9b046bc8e2af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:48:05.903876 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:05.903789    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11cca3c9-ddc6-4e64-80e7-9b046bc8e2af-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:48:06.643494 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:06.643460    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"
Jul 14 11:48:06.643726 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:06.643502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h" event={"ID":"11cca3c9-ddc6-4e64-80e7-9b046bc8e2af","Type":"ContainerDied","Data":"2b556c3fc2112622721c8ad3de747ca527fbafb7ff7d543e4735c9b4c17b3f5e"}
Jul 14 11:48:06.643726 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:06.643553    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b556c3fc2112622721c8ad3de747ca527fbafb7ff7d543e4735c9b4c17b3f5e"
Jul 14 11:48:06.675499 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:06.675463    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"]
Jul 14 11:48:06.679443 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:06.679418    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-zrp6h"]
Jul 14 11:48:06.769625 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:06.769591    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" path="/var/lib/kubelet/pods/11cca3c9-ddc6-4e64-80e7-9b046bc8e2af/volumes"
Jul 14 11:48:26.722151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:26.722119    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:48:26.727494 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:26.727473    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:48:44.895905 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.895869    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"]
Jul 14 11:48:44.896300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.896182    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:48:44.896300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.896192    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:48:44.896300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.896205    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:48:44.896300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.896210    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:48:44.896300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.896286    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:48:44.896300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.896296    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:48:44.899159 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.899142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"
Jul 14 11:48:44.901382 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.901363    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:48:44.901492 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.901436    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:48:44.906622 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.906595    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"]
Jul 14 11:48:44.924670 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:44.924634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5711c788-ff33-4fb4-9bab-949dd3498af4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7\" (UID: \"5711c788-ff33-4fb4-9bab-949dd3498af4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"
Jul 14 11:48:45.026130 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:45.026088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5711c788-ff33-4fb4-9bab-949dd3498af4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7\" (UID: \"5711c788-ff33-4fb4-9bab-949dd3498af4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"
Jul 14 11:48:45.026752 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:45.026729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5711c788-ff33-4fb4-9bab-949dd3498af4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7\" (UID: \"5711c788-ff33-4fb4-9bab-949dd3498af4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"
Jul 14 11:48:45.210331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:45.210239    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"
Jul 14 11:48:45.338201 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:45.338163    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"]
Jul 14 11:48:45.341366 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:48:45.341327    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5711c788_ff33_4fb4_9bab_949dd3498af4.slice/crio-d53be9b830541a424b8d57b709c151668fa019ed8248eec94dee35b072e2fddd WatchSource:0}: Error finding container d53be9b830541a424b8d57b709c151668fa019ed8248eec94dee35b072e2fddd: Status 404 returned error can't find the container with id d53be9b830541a424b8d57b709c151668fa019ed8248eec94dee35b072e2fddd
Jul 14 11:48:45.828660 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:45.828568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7" event={"ID":"5711c788-ff33-4fb4-9bab-949dd3498af4","Type":"ContainerStarted","Data":"ad70b941b76cde8f729d544a7b425da23569c08b5c420d47f20ddabdef5bd9fb"}
Jul 14 11:48:45.828660 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:45.828628    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7" event={"ID":"5711c788-ff33-4fb4-9bab-949dd3498af4","Type":"ContainerStarted","Data":"d53be9b830541a424b8d57b709c151668fa019ed8248eec94dee35b072e2fddd"}
Jul 14 11:48:45.845735 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:45.845664    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7" podStartSLOduration=1.8456482699999999 podStartE2EDuration="1.84564827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:48:45.844814067 +0000 UTC m=+1819.645369808" watchObservedRunningTime="2026-07-14 11:48:45.84564827 +0000 UTC m=+1819.646204010"
Jul 14 11:48:46.834768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:46.834735    2564 generic.go:356] "Generic (PLEG): container finished" podID="5711c788-ff33-4fb4-9bab-949dd3498af4" containerID="ad70b941b76cde8f729d544a7b425da23569c08b5c420d47f20ddabdef5bd9fb" exitCode=0
Jul 14 11:48:46.835157 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:46.834824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7" event={"ID":"5711c788-ff33-4fb4-9bab-949dd3498af4","Type":"ContainerDied","Data":"ad70b941b76cde8f729d544a7b425da23569c08b5c420d47f20ddabdef5bd9fb"}
Jul 14 11:48:47.966045 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:47.966018    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"
Jul 14 11:48:48.052779 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:48.052748    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5711c788-ff33-4fb4-9bab-949dd3498af4-maas-api-service-ca\") pod \"5711c788-ff33-4fb4-9bab-949dd3498af4\" (UID: \"5711c788-ff33-4fb4-9bab-949dd3498af4\") "
Jul 14 11:48:48.053083 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:48.053062    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5711c788-ff33-4fb4-9bab-949dd3498af4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5711c788-ff33-4fb4-9bab-949dd3498af4" (UID: "5711c788-ff33-4fb4-9bab-949dd3498af4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:48:48.153689 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:48.153608    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5711c788-ff33-4fb4-9bab-949dd3498af4-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:48:48.845553 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:48.845519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7" event={"ID":"5711c788-ff33-4fb4-9bab-949dd3498af4","Type":"ContainerDied","Data":"d53be9b830541a424b8d57b709c151668fa019ed8248eec94dee35b072e2fddd"}
Jul 14 11:48:48.845553 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:48.845540    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7"
Jul 14 11:48:48.845742 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:48.845566    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d53be9b830541a424b8d57b709c151668fa019ed8248eec94dee35b072e2fddd"
Jul 14 11:48:57.470490 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:48:57.470454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:50:33.474925 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:50:33.474886    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:50:40.973712 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:50:40.973658    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:50:49.572608 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:50:49.572568    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:50:59.670476 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:50:59.670432    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:51:02.203535 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.203501    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"]
Jul 14 11:51:02.203947 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.203841    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:51:02.203947 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.203851    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:51:02.203947 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.203866    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5711c788-ff33-4fb4-9bab-949dd3498af4" containerName="revoke-keys"
Jul 14 11:51:02.203947 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.203871    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5711c788-ff33-4fb4-9bab-949dd3498af4" containerName="revoke-keys"
Jul 14 11:51:02.203947 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.203931    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5711c788-ff33-4fb4-9bab-949dd3498af4" containerName="revoke-keys"
Jul 14 11:51:02.203947 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.203941    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11cca3c9-ddc6-4e64-80e7-9b046bc8e2af" containerName="revoke-keys"
Jul 14 11:51:02.206915 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.206898    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"
Jul 14 11:51:02.209512 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.209492    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:51:02.209606 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.209585    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:51:02.216634 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.216606    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"]
Jul 14 11:51:02.251977 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.251949    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6d0847-3fde-4fd3-8331-e818c3d5e648-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9\" (UID: \"dd6d0847-3fde-4fd3-8331-e818c3d5e648\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"
Jul 14 11:51:02.353222 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.353184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6d0847-3fde-4fd3-8331-e818c3d5e648-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9\" (UID: \"dd6d0847-3fde-4fd3-8331-e818c3d5e648\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"
Jul 14 11:51:02.353821 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.353804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6d0847-3fde-4fd3-8331-e818c3d5e648-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9\" (UID: \"dd6d0847-3fde-4fd3-8331-e818c3d5e648\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"
Jul 14 11:51:02.517979 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.517948    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"
Jul 14 11:51:02.655577 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:02.652513    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"]
Jul 14 11:51:03.488739 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:03.488687    2564 generic.go:356] "Generic (PLEG): container finished" podID="dd6d0847-3fde-4fd3-8331-e818c3d5e648" containerID="5575d6c9e114ce35e96a759e1f0b6cdbbf1d018f88e26d09aa622cf0f224d4ae" exitCode=0
Jul 14 11:51:03.488739 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:03.488726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9" event={"ID":"dd6d0847-3fde-4fd3-8331-e818c3d5e648","Type":"ContainerDied","Data":"5575d6c9e114ce35e96a759e1f0b6cdbbf1d018f88e26d09aa622cf0f224d4ae"}
Jul 14 11:51:03.489163 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:03.488782    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9" event={"ID":"dd6d0847-3fde-4fd3-8331-e818c3d5e648","Type":"ContainerStarted","Data":"4e8df7ee79f0caeda760e835e3e95973a7a8fb885f97115ad5c7223b677a1b18"}
Jul 14 11:51:04.618623 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:04.618596    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"
Jul 14 11:51:04.674227 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:04.674201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6d0847-3fde-4fd3-8331-e818c3d5e648-maas-api-service-ca\") pod \"dd6d0847-3fde-4fd3-8331-e818c3d5e648\" (UID: \"dd6d0847-3fde-4fd3-8331-e818c3d5e648\") "
Jul 14 11:51:04.674518 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:04.674497    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd6d0847-3fde-4fd3-8331-e818c3d5e648-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd6d0847-3fde-4fd3-8331-e818c3d5e648" (UID: "dd6d0847-3fde-4fd3-8331-e818c3d5e648"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:51:04.775826 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:04.775597    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd6d0847-3fde-4fd3-8331-e818c3d5e648-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:51:05.499722 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:05.499678    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9"
Jul 14 11:51:05.499938 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:05.499679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9" event={"ID":"dd6d0847-3fde-4fd3-8331-e818c3d5e648","Type":"ContainerDied","Data":"4e8df7ee79f0caeda760e835e3e95973a7a8fb885f97115ad5c7223b677a1b18"}
Jul 14 11:51:05.499938 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:05.499815    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e8df7ee79f0caeda760e835e3e95973a7a8fb885f97115ad5c7223b677a1b18"
Jul 14 11:51:16.069965 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:16.069925    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:51:26.794999 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:26.794972    2564 scope.go:117] "RemoveContainer" containerID="700f9d74943b4c66fe6b2489c65e07ef2218cf4e8298b43612e7c2b432b28845"
Jul 14 11:51:26.802797 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:26.802775    2564 scope.go:117] "RemoveContainer" containerID="9ffa6f8b8319a09f3c886749d4f1fb92f3f8e669604818217ef341b7e2f9e05d"
Jul 14 11:51:55.468902 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:51:55.468866    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:52:07.763435 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.763397    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"]
Jul 14 11:52:07.764032 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.763751    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd6d0847-3fde-4fd3-8331-e818c3d5e648" containerName="revoke-keys"
Jul 14 11:52:07.764032 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.763763    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd6d0847-3fde-4fd3-8331-e818c3d5e648" containerName="revoke-keys"
Jul 14 11:52:07.764032 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.763824    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd6d0847-3fde-4fd3-8331-e818c3d5e648" containerName="revoke-keys"
Jul 14 11:52:07.766849 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.766819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:07.769331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.769307    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-67b298da\"/\"kube-root-ca.crt\""
Jul 14 11:52:07.769331 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.769322    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-67b298da\"/\"openshift-service-ca.crt\""
Jul 14 11:52:07.769890 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.769872    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-67b298da\"/\"default-dockercfg-px56r\""
Jul 14 11:52:07.769982 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.769926    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-67b298da\"/\"e2e-lifecycle-model-67b298da-kserve-self-signed-certs\""
Jul 14 11:52:07.775563 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.775541    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"]
Jul 14 11:52:07.943919 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.943883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-dshm\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:07.943919 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.943916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tmp-dir\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:07.944239 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.944002    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkrf5\" (UniqueName: \"kubernetes.io/projected/417f9905-9ac2-46ba-a4cd-48b57820c0c2-kube-api-access-xkrf5\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:07.944239 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.944061    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-home\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:07.944318 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.944286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-model-cache\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:07.944354 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:07.944317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tls-certs\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045391 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-model-cache\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045391 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tls-certs\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045613 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045408    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-dshm\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045613 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tmp-dir\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045613 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045456    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkrf5\" (UniqueName: \"kubernetes.io/projected/417f9905-9ac2-46ba-a4cd-48b57820c0c2-kube-api-access-xkrf5\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045613 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-home\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045901 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045872    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-model-cache\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045974 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045899    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tmp-dir\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.045974 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.045924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-home\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.047689 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.047666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-dshm\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.047935 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.047918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tls-certs\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.061587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.061564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkrf5\" (UniqueName: \"kubernetes.io/projected/417f9905-9ac2-46ba-a4cd-48b57820c0c2-kube-api-access-xkrf5\") pod \"e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") " pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.077562 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.077537    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:08.208242 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.208216    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"]
Jul 14 11:52:08.209840 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:52:08.209811    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod417f9905_9ac2_46ba_a4cd_48b57820c0c2.slice/crio-fa14795295be19959b2cdfcff38397395d60a2d311a123816cc5007391d412eb WatchSource:0}: Error finding container fa14795295be19959b2cdfcff38397395d60a2d311a123816cc5007391d412eb: Status 404 returned error can't find the container with id fa14795295be19959b2cdfcff38397395d60a2d311a123816cc5007391d412eb
Jul 14 11:52:08.799858 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.799820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z" event={"ID":"417f9905-9ac2-46ba-a4cd-48b57820c0c2","Type":"ContainerStarted","Data":"20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef"}
Jul 14 11:52:08.799858 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.799861    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z" event={"ID":"417f9905-9ac2-46ba-a4cd-48b57820c0c2","Type":"ContainerStarted","Data":"fa14795295be19959b2cdfcff38397395d60a2d311a123816cc5007391d412eb"}
Jul 14 11:52:08.820148 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:08.820095    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z" podStartSLOduration=1.820079297 podStartE2EDuration="1.820079297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:52:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:52:08.817817569 +0000 UTC m=+2022.618373308" watchObservedRunningTime="2026-07-14 11:52:08.820079297 +0000 UTC m=+2022.620635036"
Jul 14 11:52:10.866585 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:10.866550    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:52:18.078662 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:18.078628    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:18.078662 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:18.078672    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:18.091332 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:18.091306    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:18.857016 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:18.856988    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:52:40.569574 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:40.569538    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:52:43.607872 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.607826    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"]
Jul 14 11:52:43.611417 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.611393    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"
Jul 14 11:52:43.613859 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.613827    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:52:43.613971 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.613861    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:52:43.619502 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.619482    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"]
Jul 14 11:52:43.738762 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.738726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a434d7d4-345a-440f-8af0-03604a710d3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-67b298da-2dhxl\" (UID: \"a434d7d4-345a-440f-8af0-03604a710d3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"
Jul 14 11:52:43.840093 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.840054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a434d7d4-345a-440f-8af0-03604a710d3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-67b298da-2dhxl\" (UID: \"a434d7d4-345a-440f-8af0-03604a710d3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"
Jul 14 11:52:43.840717 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.840674    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a434d7d4-345a-440f-8af0-03604a710d3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-67b298da-2dhxl\" (UID: \"a434d7d4-345a-440f-8af0-03604a710d3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"
Jul 14 11:52:43.922137 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:43.922061    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"
Jul 14 11:52:44.054184 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:44.054157    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"]
Jul 14 11:52:44.056671 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:52:44.056645    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda434d7d4_345a_440f_8af0_03604a710d3d.slice/crio-5fb81b7d56eb7b35809dfde6d3198bcc0ba10e6cb02349844bed04fbc06254a7 WatchSource:0}: Error finding container 5fb81b7d56eb7b35809dfde6d3198bcc0ba10e6cb02349844bed04fbc06254a7: Status 404 returned error can't find the container with id 5fb81b7d56eb7b35809dfde6d3198bcc0ba10e6cb02349844bed04fbc06254a7
Jul 14 11:52:44.972939 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:44.972849    2564 generic.go:356] "Generic (PLEG): container finished" podID="a434d7d4-345a-440f-8af0-03604a710d3d" containerID="9c6a58bcb05f92b37d6fba984be36f4ced2d4b5e562c32fb93ce0194f8fe39d9" exitCode=0
Jul 14 11:52:44.972939 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:44.972921    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl" event={"ID":"a434d7d4-345a-440f-8af0-03604a710d3d","Type":"ContainerDied","Data":"9c6a58bcb05f92b37d6fba984be36f4ced2d4b5e562c32fb93ce0194f8fe39d9"}
Jul 14 11:52:44.973351 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:44.972970    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl" event={"ID":"a434d7d4-345a-440f-8af0-03604a710d3d","Type":"ContainerStarted","Data":"5fb81b7d56eb7b35809dfde6d3198bcc0ba10e6cb02349844bed04fbc06254a7"}
Jul 14 11:52:46.109134 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:46.109109    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"
Jul 14 11:52:46.160856 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:46.160820    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a434d7d4-345a-440f-8af0-03604a710d3d-maas-api-service-ca\") pod \"a434d7d4-345a-440f-8af0-03604a710d3d\" (UID: \"a434d7d4-345a-440f-8af0-03604a710d3d\") "
Jul 14 11:52:46.161152 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:46.161131    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a434d7d4-345a-440f-8af0-03604a710d3d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a434d7d4-345a-440f-8af0-03604a710d3d" (UID: "a434d7d4-345a-440f-8af0-03604a710d3d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:52:46.262005 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:46.261925    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a434d7d4-345a-440f-8af0-03604a710d3d-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:52:46.983283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:46.983245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl" event={"ID":"a434d7d4-345a-440f-8af0-03604a710d3d","Type":"ContainerDied","Data":"5fb81b7d56eb7b35809dfde6d3198bcc0ba10e6cb02349844bed04fbc06254a7"}
Jul 14 11:52:46.983283 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:46.983280    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-67b298da-2dhxl"
Jul 14 11:52:46.983485 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:46.983295    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fb81b7d56eb7b35809dfde6d3198bcc0ba10e6cb02349844bed04fbc06254a7"
Jul 14 11:52:58.102376 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:52:58.102342    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:53:03.686551 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:03.686511    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"]
Jul 14 11:53:18.706851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:18.706801    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z" podUID="417f9905-9ac2-46ba-a4cd-48b57820c0c2" containerName="main" containerID="cri-o://20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef" gracePeriod=45
Jul 14 11:53:18.846733 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:18.846667    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z" podUID="417f9905-9ac2-46ba-a4cd-48b57820c0c2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": dial tcp 10.134.0.40:8000: connect: connection refused"
Jul 14 11:53:18.955020 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:18.954989    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:53:19.038384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038348    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-dshm\") pod \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") "
Jul 14 11:53:19.038384 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038383    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tls-certs\") pod \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") "
Jul 14 11:53:19.038603 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038404    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tmp-dir\") pod \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") "
Jul 14 11:53:19.038603 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038466    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkrf5\" (UniqueName: \"kubernetes.io/projected/417f9905-9ac2-46ba-a4cd-48b57820c0c2-kube-api-access-xkrf5\") pod \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") "
Jul 14 11:53:19.038603 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038488    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-model-cache\") pod \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") "
Jul 14 11:53:19.038603 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038507    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-home\") pod \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\" (UID: \"417f9905-9ac2-46ba-a4cd-48b57820c0c2\") "
Jul 14 11:53:19.038833 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038755    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "417f9905-9ac2-46ba-a4cd-48b57820c0c2" (UID: "417f9905-9ac2-46ba-a4cd-48b57820c0c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:53:19.038945 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038923    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-home" (OuterVolumeSpecName: "home") pod "417f9905-9ac2-46ba-a4cd-48b57820c0c2" (UID: "417f9905-9ac2-46ba-a4cd-48b57820c0c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:53:19.038998 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.038937    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "417f9905-9ac2-46ba-a4cd-48b57820c0c2" (UID: "417f9905-9ac2-46ba-a4cd-48b57820c0c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:53:19.041043 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.041016    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-dshm" (OuterVolumeSpecName: "dshm") pod "417f9905-9ac2-46ba-a4cd-48b57820c0c2" (UID: "417f9905-9ac2-46ba-a4cd-48b57820c0c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:53:19.041161 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.041115    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "417f9905-9ac2-46ba-a4cd-48b57820c0c2" (UID: "417f9905-9ac2-46ba-a4cd-48b57820c0c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:53:19.041161 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.041123    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/417f9905-9ac2-46ba-a4cd-48b57820c0c2-kube-api-access-xkrf5" (OuterVolumeSpecName: "kube-api-access-xkrf5") pod "417f9905-9ac2-46ba-a4cd-48b57820c0c2" (UID: "417f9905-9ac2-46ba-a4cd-48b57820c0c2"). InnerVolumeSpecName "kube-api-access-xkrf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:53:19.137910 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.137869    2564 generic.go:356] "Generic (PLEG): container finished" podID="417f9905-9ac2-46ba-a4cd-48b57820c0c2" containerID="20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef" exitCode=0
Jul 14 11:53:19.137910 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.137913    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"
Jul 14 11:53:19.138134 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.137958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z" event={"ID":"417f9905-9ac2-46ba-a4cd-48b57820c0c2","Type":"ContainerDied","Data":"20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef"}
Jul 14 11:53:19.138134 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.137998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z" event={"ID":"417f9905-9ac2-46ba-a4cd-48b57820c0c2","Type":"ContainerDied","Data":"fa14795295be19959b2cdfcff38397395d60a2d311a123816cc5007391d412eb"}
Jul 14 11:53:19.138134 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.138016    2564 scope.go:117] "RemoveContainer" containerID="20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef"
Jul 14 11:53:19.139183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.139048    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:53:19.139183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.139066    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:53:19.139183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.139096    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:53:19.139183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.139106    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkrf5\" (UniqueName: \"kubernetes.io/projected/417f9905-9ac2-46ba-a4cd-48b57820c0c2-kube-api-access-xkrf5\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:53:19.139183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.139116    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:53:19.139183 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.139125    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/417f9905-9ac2-46ba-a4cd-48b57820c0c2-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:53:19.146866 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.146850    2564 scope.go:117] "RemoveContainer" containerID="20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef"
Jul 14 11:53:19.147126 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:53:19.147108    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef\": container with ID starting with 20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef not found: ID does not exist" containerID="20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef"
Jul 14 11:53:19.147180 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.147135    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef"} err="failed to get container status \"20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef\": rpc error: code = NotFound desc = could not find container \"20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef\": container with ID starting with 20e2b1ba8ca1f2399092b23a4e6cdf17edb067d4bd2effc5295e2af149308cef not found: ID does not exist"
Jul 14 11:53:19.159645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.159622    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"]
Jul 14 11:53:19.164753 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:19.164733    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-67b298da/e2e-lifecycle-model-67b298da-kserve-56cb776bc9-r8f9z"]
Jul 14 11:53:20.769003 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:20.768968    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="417f9905-9ac2-46ba-a4cd-48b57820c0c2" path="/var/lib/kubelet/pods/417f9905-9ac2-46ba-a4cd-48b57820c0c2/volumes"
Jul 14 11:53:26.751510 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:26.751485    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:53:26.757768 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:26.757748    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:53:26.832437 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:26.832407    2564 scope.go:117] "RemoveContainer" containerID="c38c42fa8ddb262be9b47c35b17bf70a60fc45ab7e86452a5b2818f41a58c8d9"
Jul 14 11:53:54.682714 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:53:54.682667    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:54:09.159451 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:54:09.159415    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:54:26.853630 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:54:26.853599    2564 scope.go:117] "RemoveContainer" containerID="c7f05a83fd1002edeeefd63ff74173cbd7bfc3bf5bded7281ba10a1ba36907d5"
Jul 14 11:54:57.898161 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:54:57.898125    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:55:09.268939 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:09.268904    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:55:15.251619 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.251585    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"]
Jul 14 11:55:15.252022 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.251935    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="417f9905-9ac2-46ba-a4cd-48b57820c0c2" containerName="main"
Jul 14 11:55:15.252022 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.251946    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="417f9905-9ac2-46ba-a4cd-48b57820c0c2" containerName="main"
Jul 14 11:55:15.252022 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.251956    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a434d7d4-345a-440f-8af0-03604a710d3d" containerName="revoke-keys"
Jul 14 11:55:15.252022 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.251961    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a434d7d4-345a-440f-8af0-03604a710d3d" containerName="revoke-keys"
Jul 14 11:55:15.252151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.252037    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="417f9905-9ac2-46ba-a4cd-48b57820c0c2" containerName="main"
Jul 14 11:55:15.252151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.252048    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a434d7d4-345a-440f-8af0-03604a710d3d" containerName="revoke-keys"
Jul 14 11:55:15.255238 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.255220    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.258272 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.258252    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8567a3\"/\"test-model-8567a3-kserve-self-signed-certs\""
Jul 14 11:55:15.258272 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.258268    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8567a3\"/\"kube-root-ca.crt\""
Jul 14 11:55:15.258465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.258268    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8567a3\"/\"default-dockercfg-h6qzr\""
Jul 14 11:55:15.258465 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.258313    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8567a3\"/\"openshift-service-ca.crt\""
Jul 14 11:55:15.265189 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.265165    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"]
Jul 14 11:55:15.306340 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.306311    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-home\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.306482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.306356    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tls-certs\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.306482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.306382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-dshm\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.306482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.306398    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tmp-dir\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.306482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.306417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zvxz\" (UniqueName: \"kubernetes.io/projected/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-kube-api-access-7zvxz\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.306482 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.306451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-model-cache\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407132 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-home\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407144    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tls-certs\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407189    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-dshm\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tmp-dir\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407304 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zvxz\" (UniqueName: \"kubernetes.io/projected/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-kube-api-access-7zvxz\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407501 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407377    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-model-cache\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407575 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407551    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-home\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407620 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tmp-dir\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.407760 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.407741    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-model-cache\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.409512 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.409488    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-dshm\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.409774 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.409758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tls-certs\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.415501 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.415483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zvxz\" (UniqueName: \"kubernetes.io/projected/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-kube-api-access-7zvxz\") pod \"test-model-8567a3-kserve-556b85c757-7s49c\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") " pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.564818 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.564773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:15.693626 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:15.693548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"]
Jul 14 11:55:15.695912 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:55:15.695885    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dbea8ed_8bed_4d5a_90c9_f7a30433631a.slice/crio-3c590f67042b983b6eaf656bf3e8c4577f2965c3f72e51dac8bd3e4f06cc0c1a WatchSource:0}: Error finding container 3c590f67042b983b6eaf656bf3e8c4577f2965c3f72e51dac8bd3e4f06cc0c1a: Status 404 returned error can't find the container with id 3c590f67042b983b6eaf656bf3e8c4577f2965c3f72e51dac8bd3e4f06cc0c1a
Jul 14 11:55:16.684748 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:16.684714    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c" event={"ID":"3dbea8ed-8bed-4d5a-90c9-f7a30433631a","Type":"ContainerStarted","Data":"5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0"}
Jul 14 11:55:16.685134 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:16.684756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c" event={"ID":"3dbea8ed-8bed-4d5a-90c9-f7a30433631a","Type":"ContainerStarted","Data":"3c590f67042b983b6eaf656bf3e8c4577f2965c3f72e51dac8bd3e4f06cc0c1a"}
Jul 14 11:55:16.701893 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:16.701839    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c" podStartSLOduration=1.701823539 podStartE2EDuration="1.701823539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:55:16.700199716 +0000 UTC m=+2210.500755482" watchObservedRunningTime="2026-07-14 11:55:16.701823539 +0000 UTC m=+2210.502379280"
Jul 14 11:55:21.466591 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:21.466555    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:55:25.565729 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:25.565657    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:25.566145 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:25.565751    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:25.578810 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:25.578780    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:25.738268 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:25.738232    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:55:51.077816 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.077720    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"]
Jul 14 11:55:51.081415 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.081389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.083901 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.083868    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-738ce7\"/\"kube-root-ca.crt\""
Jul 14 11:55:51.083901 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.083868    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-738ce7\"/\"openshift-service-ca.crt\""
Jul 14 11:55:51.084607 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.084584    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-738ce7\"/\"test-model-738ce7-kserve-self-signed-certs\""
Jul 14 11:55:51.091103 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.091065    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-738ce7\"/\"default-dockercfg-tgktz\""
Jul 14 11:55:51.095602 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.095573    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"]
Jul 14 11:55:51.229041 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.229003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-model-cache\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.229041 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.229047    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99zr9\" (UniqueName: \"kubernetes.io/projected/fa376820-4063-4fd6-b5d9-a97dd54a9c98-kube-api-access-99zr9\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.229315 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.229082    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-home\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.229315 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.229136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-dshm\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.229315 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.229161    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tmp-dir\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.229315 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.229184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tls-certs\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.330607 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.330501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-model-cache\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.330607 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.330546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99zr9\" (UniqueName: \"kubernetes.io/projected/fa376820-4063-4fd6-b5d9-a97dd54a9c98-kube-api-access-99zr9\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.330607 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.330571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-home\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.330607 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.330608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-dshm\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.331011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.330623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tmp-dir\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.331011 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.330641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tls-certs\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.331127 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.331026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-model-cache\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.331127 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.331107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tmp-dir\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.331216 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.331080    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-home\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.333008 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.332980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-dshm\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.333125 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.333109    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tls-certs\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.338982 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.338963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99zr9\" (UniqueName: \"kubernetes.io/projected/fa376820-4063-4fd6-b5d9-a97dd54a9c98-kube-api-access-99zr9\") pod \"test-model-738ce7-kserve-dd95b6d9f-xmfdw\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") " pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.397961 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.397906    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:55:51.533967 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.533900    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"]
Jul 14 11:55:51.536849 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:55:51.536824    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa376820_4063_4fd6_b5d9_a97dd54a9c98.slice/crio-fb8301737ff8522c0b72a0b130ec2172cc1084108bbd9459c6399f6493d67a33 WatchSource:0}: Error finding container fb8301737ff8522c0b72a0b130ec2172cc1084108bbd9459c6399f6493d67a33: Status 404 returned error can't find the container with id fb8301737ff8522c0b72a0b130ec2172cc1084108bbd9459c6399f6493d67a33
Jul 14 11:55:51.855170 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.855081    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw" event={"ID":"fa376820-4063-4fd6-b5d9-a97dd54a9c98","Type":"ContainerStarted","Data":"50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d"}
Jul 14 11:55:51.855170 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.855121    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw" event={"ID":"fa376820-4063-4fd6-b5d9-a97dd54a9c98","Type":"ContainerStarted","Data":"fb8301737ff8522c0b72a0b130ec2172cc1084108bbd9459c6399f6493d67a33"}
Jul 14 11:55:51.874211 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:51.874146    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw" podStartSLOduration=0.874124525 podStartE2EDuration="874.124525ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:55:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:55:51.871539949 +0000 UTC m=+2245.672095688" watchObservedRunningTime="2026-07-14 11:55:51.874124525 +0000 UTC m=+2245.674680265"
Jul 14 11:55:55.672677 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:55:55.672630    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:56:01.398708 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:01.398639    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:56:01.399203 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:01.398728    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:56:01.414145 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:01.414112    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:56:01.915101 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:01.915064    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:56:09.362097 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:09.362050    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:56:11.265415 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:11.265382    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"]
Jul 14 11:56:12.172368 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.172329    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"]
Jul 14 11:56:12.327456 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.327424    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"]
Jul 14 11:56:12.330766 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.330748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"
Jul 14 11:56:12.333026 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.333003    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:56:12.333141 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.333006    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:56:12.337709 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.337670    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"]
Jul 14 11:56:12.430545 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.430444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee2cd1b-8389-4fa4-a6eb-d8a430fa3013-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9\" (UID: \"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"
Jul 14 11:56:12.531544 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.531499    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee2cd1b-8389-4fa4-a6eb-d8a430fa3013-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9\" (UID: \"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"
Jul 14 11:56:12.532150 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.532126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee2cd1b-8389-4fa4-a6eb-d8a430fa3013-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9\" (UID: \"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"
Jul 14 11:56:12.644572 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.644534    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"
Jul 14 11:56:12.781671 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.781647    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"]
Jul 14 11:56:12.784198 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:56:12.784155    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee2cd1b_8389_4fa4_a6eb_d8a430fa3013.slice/crio-5ecf9807343baf6faaa54d087cde92ea8ad8630bb0c312bf0d9bb8c12fd261c1 WatchSource:0}: Error finding container 5ecf9807343baf6faaa54d087cde92ea8ad8630bb0c312bf0d9bb8c12fd261c1: Status 404 returned error can't find the container with id 5ecf9807343baf6faaa54d087cde92ea8ad8630bb0c312bf0d9bb8c12fd261c1
Jul 14 11:56:12.958487 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:12.958395    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9" event={"ID":"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013","Type":"ContainerStarted","Data":"5ecf9807343baf6faaa54d087cde92ea8ad8630bb0c312bf0d9bb8c12fd261c1"}
Jul 14 11:56:13.964739 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:13.964679    2564 generic.go:356] "Generic (PLEG): container finished" podID="aee2cd1b-8389-4fa4-a6eb-d8a430fa3013" containerID="d3e6df551b229b435fb3a29894a5291fbac98d96673e58c9aa76dfbdd8a55f3b" exitCode=0
Jul 14 11:56:13.965119 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:13.964752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9" event={"ID":"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013","Type":"ContainerDied","Data":"d3e6df551b229b435fb3a29894a5291fbac98d96673e58c9aa76dfbdd8a55f3b"}
Jul 14 11:56:15.100440 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:15.100414    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"
Jul 14 11:56:15.159360 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:15.159325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee2cd1b-8389-4fa4-a6eb-d8a430fa3013-maas-api-service-ca\") pod \"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013\" (UID: \"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013\") "
Jul 14 11:56:15.159691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:15.159668    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee2cd1b-8389-4fa4-a6eb-d8a430fa3013-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aee2cd1b-8389-4fa4-a6eb-d8a430fa3013" (UID: "aee2cd1b-8389-4fa4-a6eb-d8a430fa3013"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:56:15.260355 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:15.260279    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee2cd1b-8389-4fa4-a6eb-d8a430fa3013-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:15.975562 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:15.975539    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9"
Jul 14 11:56:15.975778 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:15.975537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9" event={"ID":"aee2cd1b-8389-4fa4-a6eb-d8a430fa3013","Type":"ContainerDied","Data":"5ecf9807343baf6faaa54d087cde92ea8ad8630bb0c312bf0d9bb8c12fd261c1"}
Jul 14 11:56:15.975778 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:15.975656    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ecf9807343baf6faaa54d087cde92ea8ad8630bb0c312bf0d9bb8c12fd261c1"
Jul 14 11:56:21.767385 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:21.767349    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:56:26.285639 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.285599    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c" podUID="3dbea8ed-8bed-4d5a-90c9-f7a30433631a" containerName="main" containerID="cri-o://5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0" gracePeriod=45
Jul 14 11:56:26.531500 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.531476    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:56:26.668450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668352    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-home\") pod \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") "
Jul 14 11:56:26.668450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668397    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-dshm\") pod \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") "
Jul 14 11:56:26.668450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zvxz\" (UniqueName: \"kubernetes.io/projected/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-kube-api-access-7zvxz\") pod \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") "
Jul 14 11:56:26.668450 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668446    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tls-certs\") pod \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") "
Jul 14 11:56:26.668815 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668500    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tmp-dir\") pod \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") "
Jul 14 11:56:26.668815 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668530    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-model-cache\") pod \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\" (UID: \"3dbea8ed-8bed-4d5a-90c9-f7a30433631a\") "
Jul 14 11:56:26.668815 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668674    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-home" (OuterVolumeSpecName: "home") pod "3dbea8ed-8bed-4d5a-90c9-f7a30433631a" (UID: "3dbea8ed-8bed-4d5a-90c9-f7a30433631a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:26.668815 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668777    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3dbea8ed-8bed-4d5a-90c9-f7a30433631a" (UID: "3dbea8ed-8bed-4d5a-90c9-f7a30433631a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:26.668960 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668838    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:26.668960 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.668876    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-model-cache" (OuterVolumeSpecName: "model-cache") pod "3dbea8ed-8bed-4d5a-90c9-f7a30433631a" (UID: "3dbea8ed-8bed-4d5a-90c9-f7a30433631a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:26.670645 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.670623    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-kube-api-access-7zvxz" (OuterVolumeSpecName: "kube-api-access-7zvxz") pod "3dbea8ed-8bed-4d5a-90c9-f7a30433631a" (UID: "3dbea8ed-8bed-4d5a-90c9-f7a30433631a"). InnerVolumeSpecName "kube-api-access-7zvxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:56:26.670736 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.670669    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3dbea8ed-8bed-4d5a-90c9-f7a30433631a" (UID: "3dbea8ed-8bed-4d5a-90c9-f7a30433631a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:56:26.670736 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.670688    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-dshm" (OuterVolumeSpecName: "dshm") pod "3dbea8ed-8bed-4d5a-90c9-f7a30433631a" (UID: "3dbea8ed-8bed-4d5a-90c9-f7a30433631a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:26.769994 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.769960    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:26.769994 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.769993    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zvxz\" (UniqueName: \"kubernetes.io/projected/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-kube-api-access-7zvxz\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:26.770252 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.770010    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:26.770252 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.770045    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:26.770252 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:26.770064    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dbea8ed-8bed-4d5a-90c9-f7a30433631a-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:27.028765 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.028726    2564 generic.go:356] "Generic (PLEG): container finished" podID="3dbea8ed-8bed-4d5a-90c9-f7a30433631a" containerID="5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0" exitCode=0
Jul 14 11:56:27.028954 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.028785    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"
Jul 14 11:56:27.028954 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.028815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c" event={"ID":"3dbea8ed-8bed-4d5a-90c9-f7a30433631a","Type":"ContainerDied","Data":"5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0"}
Jul 14 11:56:27.028954 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.028863    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c" event={"ID":"3dbea8ed-8bed-4d5a-90c9-f7a30433631a","Type":"ContainerDied","Data":"3c590f67042b983b6eaf656bf3e8c4577f2965c3f72e51dac8bd3e4f06cc0c1a"}
Jul 14 11:56:27.028954 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.028884    2564 scope.go:117] "RemoveContainer" containerID="5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0"
Jul 14 11:56:27.037144 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.037125    2564 scope.go:117] "RemoveContainer" containerID="5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0"
Jul 14 11:56:27.037410 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:56:27.037390    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0\": container with ID starting with 5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0 not found: ID does not exist" containerID="5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0"
Jul 14 11:56:27.037474 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.037420    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0"} err="failed to get container status \"5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0\": rpc error: code = NotFound desc = could not find container \"5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0\": container with ID starting with 5355c388138fdd01ae2a7db2d66c8b3d17dd8dd405a154b0ccb885faa9aff7b0 not found: ID does not exist"
Jul 14 11:56:27.044543 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.044520    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"]
Jul 14 11:56:27.048488 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.048463    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8567a3/test-model-8567a3-kserve-556b85c757-7s49c"]
Jul 14 11:56:27.193837 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.193799    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw" podUID="fa376820-4063-4fd6-b5d9-a97dd54a9c98" containerName="main" containerID="cri-o://50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d" gracePeriod=45
Jul 14 11:56:27.443080 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.443056    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:56:27.576516 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576424    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-model-cache\") pod \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") "
Jul 14 11:56:27.576516 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576464    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-dshm\") pod \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") "
Jul 14 11:56:27.576516 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576498    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tmp-dir\") pod \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") "
Jul 14 11:56:27.576851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576545    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-home\") pod \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") "
Jul 14 11:56:27.576851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576566    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tls-certs\") pod \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") "
Jul 14 11:56:27.576851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576585    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99zr9\" (UniqueName: \"kubernetes.io/projected/fa376820-4063-4fd6-b5d9-a97dd54a9c98-kube-api-access-99zr9\") pod \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\" (UID: \"fa376820-4063-4fd6-b5d9-a97dd54a9c98\") "
Jul 14 11:56:27.576851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576769    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa376820-4063-4fd6-b5d9-a97dd54a9c98" (UID: "fa376820-4063-4fd6-b5d9-a97dd54a9c98"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:27.576851 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576841    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa376820-4063-4fd6-b5d9-a97dd54a9c98" (UID: "fa376820-4063-4fd6-b5d9-a97dd54a9c98"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:27.577033 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.576856    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-home" (OuterVolumeSpecName: "home") pod "fa376820-4063-4fd6-b5d9-a97dd54a9c98" (UID: "fa376820-4063-4fd6-b5d9-a97dd54a9c98"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:27.578738 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.578690    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa376820-4063-4fd6-b5d9-a97dd54a9c98" (UID: "fa376820-4063-4fd6-b5d9-a97dd54a9c98"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 11:56:27.578738 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.578722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-dshm" (OuterVolumeSpecName: "dshm") pod "fa376820-4063-4fd6-b5d9-a97dd54a9c98" (UID: "fa376820-4063-4fd6-b5d9-a97dd54a9c98"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 11:56:27.578896 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.578773    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa376820-4063-4fd6-b5d9-a97dd54a9c98-kube-api-access-99zr9" (OuterVolumeSpecName: "kube-api-access-99zr9") pod "fa376820-4063-4fd6-b5d9-a97dd54a9c98" (UID: "fa376820-4063-4fd6-b5d9-a97dd54a9c98"). InnerVolumeSpecName "kube-api-access-99zr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 11:56:27.677995 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.677956    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:27.677995 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.677989    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99zr9\" (UniqueName: \"kubernetes.io/projected/fa376820-4063-4fd6-b5d9-a97dd54a9c98-kube-api-access-99zr9\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:27.677995 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.677999    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:27.677995 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.678007    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:27.678288 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.678015    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:27.678288 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:27.678022    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa376820-4063-4fd6-b5d9-a97dd54a9c98-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:28.035200 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.035172    2564 generic.go:356] "Generic (PLEG): container finished" podID="fa376820-4063-4fd6-b5d9-a97dd54a9c98" containerID="50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d" exitCode=0
Jul 14 11:56:28.035376 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.035214    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"
Jul 14 11:56:28.035376 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.035261    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw" event={"ID":"fa376820-4063-4fd6-b5d9-a97dd54a9c98","Type":"ContainerDied","Data":"50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d"}
Jul 14 11:56:28.035376 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.035304    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw" event={"ID":"fa376820-4063-4fd6-b5d9-a97dd54a9c98","Type":"ContainerDied","Data":"fb8301737ff8522c0b72a0b130ec2172cc1084108bbd9459c6399f6493d67a33"}
Jul 14 11:56:28.035376 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.035322    2564 scope.go:117] "RemoveContainer" containerID="50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d"
Jul 14 11:56:28.044651 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.044635    2564 scope.go:117] "RemoveContainer" containerID="50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d"
Jul 14 11:56:28.044956 ip-10-0-142-172 kubenswrapper[2564]: E0714 11:56:28.044937    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d\": container with ID starting with 50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d not found: ID does not exist" containerID="50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d"
Jul 14 11:56:28.045021 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.044962    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d"} err="failed to get container status \"50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d\": rpc error: code = NotFound desc = could not find container \"50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d\": container with ID starting with 50962d7f8bbdc369560dc36ea01a552ea3759f607f6512802bad61925d5ab89d not found: ID does not exist"
Jul 14 11:56:28.066131 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.066103    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"]
Jul 14 11:56:28.073087 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.073038    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-738ce7/test-model-738ce7-kserve-dd95b6d9f-xmfdw"]
Jul 14 11:56:28.769253 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.769218    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dbea8ed-8bed-4d5a-90c9-f7a30433631a" path="/var/lib/kubelet/pods/3dbea8ed-8bed-4d5a-90c9-f7a30433631a/volumes"
Jul 14 11:56:28.769690 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:28.769590    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa376820-4063-4fd6-b5d9-a97dd54a9c98" path="/var/lib/kubelet/pods/fa376820-4063-4fd6-b5d9-a97dd54a9c98/volumes"
Jul 14 11:56:31.959881 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:31.959847    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:56:38.573299 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573265    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"]
Jul 14 11:56:38.573667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573601    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee2cd1b-8389-4fa4-a6eb-d8a430fa3013" containerName="revoke-keys"
Jul 14 11:56:38.573667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573612    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee2cd1b-8389-4fa4-a6eb-d8a430fa3013" containerName="revoke-keys"
Jul 14 11:56:38.573667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573621    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa376820-4063-4fd6-b5d9-a97dd54a9c98" containerName="main"
Jul 14 11:56:38.573667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573628    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa376820-4063-4fd6-b5d9-a97dd54a9c98" containerName="main"
Jul 14 11:56:38.573667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573635    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dbea8ed-8bed-4d5a-90c9-f7a30433631a" containerName="main"
Jul 14 11:56:38.573667 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573640    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dbea8ed-8bed-4d5a-90c9-f7a30433631a" containerName="main"
Jul 14 11:56:38.573879 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573726    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee2cd1b-8389-4fa4-a6eb-d8a430fa3013" containerName="revoke-keys"
Jul 14 11:56:38.573879 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573735    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dbea8ed-8bed-4d5a-90c9-f7a30433631a" containerName="main"
Jul 14 11:56:38.573879 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.573742    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa376820-4063-4fd6-b5d9-a97dd54a9c98" containerName="main"
Jul 14 11:56:38.576557 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.576538    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"
Jul 14 11:56:38.579921 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.579903    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:56:38.580351 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.580334    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:56:38.596837 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.596806    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"]
Jul 14 11:56:38.667290 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.667254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff35806-07ce-4530-9d2d-24d1162228af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb\" (UID: \"fff35806-07ce-4530-9d2d-24d1162228af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"
Jul 14 11:56:38.767793 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.767762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff35806-07ce-4530-9d2d-24d1162228af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb\" (UID: \"fff35806-07ce-4530-9d2d-24d1162228af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"
Jul 14 11:56:38.768459 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.768427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff35806-07ce-4530-9d2d-24d1162228af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb\" (UID: \"fff35806-07ce-4530-9d2d-24d1162228af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"
Jul 14 11:56:38.886211 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:38.886134    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"
Jul 14 11:56:39.026789 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:39.026766    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"]
Jul 14 11:56:39.028441 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:56:39.028407    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff35806_07ce_4530_9d2d_24d1162228af.slice/crio-7b4e38b77aca151b44b3f5e9d99e9d32a6add8dfb2bb36c40334a850db119812 WatchSource:0}: Error finding container 7b4e38b77aca151b44b3f5e9d99e9d32a6add8dfb2bb36c40334a850db119812: Status 404 returned error can't find the container with id 7b4e38b77aca151b44b3f5e9d99e9d32a6add8dfb2bb36c40334a850db119812
Jul 14 11:56:39.088983 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:39.088924    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb" event={"ID":"fff35806-07ce-4530-9d2d-24d1162228af","Type":"ContainerStarted","Data":"7b4e38b77aca151b44b3f5e9d99e9d32a6add8dfb2bb36c40334a850db119812"}
Jul 14 11:56:40.094757 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:40.094721    2564 generic.go:356] "Generic (PLEG): container finished" podID="fff35806-07ce-4530-9d2d-24d1162228af" containerID="dd8dec21262edeceade377e85ca47dfc05f6be329e79a0c8f1bd09028b974ca4" exitCode=0
Jul 14 11:56:40.095162 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:40.094796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb" event={"ID":"fff35806-07ce-4530-9d2d-24d1162228af","Type":"ContainerDied","Data":"dd8dec21262edeceade377e85ca47dfc05f6be329e79a0c8f1bd09028b974ca4"}
Jul 14 11:56:41.222492 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:41.222470    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"
Jul 14 11:56:41.288120 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:41.288083    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff35806-07ce-4530-9d2d-24d1162228af-maas-api-service-ca\") pod \"fff35806-07ce-4530-9d2d-24d1162228af\" (UID: \"fff35806-07ce-4530-9d2d-24d1162228af\") "
Jul 14 11:56:41.288475 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:41.288452    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fff35806-07ce-4530-9d2d-24d1162228af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fff35806-07ce-4530-9d2d-24d1162228af" (UID: "fff35806-07ce-4530-9d2d-24d1162228af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:56:41.388844 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:41.388765    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff35806-07ce-4530-9d2d-24d1162228af-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:56:42.104431 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:42.104392    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb" event={"ID":"fff35806-07ce-4530-9d2d-24d1162228af","Type":"ContainerDied","Data":"7b4e38b77aca151b44b3f5e9d99e9d32a6add8dfb2bb36c40334a850db119812"}
Jul 14 11:56:42.104605 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:42.104468    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b4e38b77aca151b44b3f5e9d99e9d32a6add8dfb2bb36c40334a850db119812"
Jul 14 11:56:42.104605 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:42.104406    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb"
Jul 14 11:56:52.476114 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:56:52.476075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:57:13.973394 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:13.973362    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:57:27.277766 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:27.277727    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:57:28.792955 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.792922    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"]
Jul 14 11:57:28.793334 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.793282    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff35806-07ce-4530-9d2d-24d1162228af" containerName="revoke-keys"
Jul 14 11:57:28.793334 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.793294    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff35806-07ce-4530-9d2d-24d1162228af" containerName="revoke-keys"
Jul 14 11:57:28.793414 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.793343    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff35806-07ce-4530-9d2d-24d1162228af" containerName="revoke-keys"
Jul 14 11:57:28.798100 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.798083    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"
Jul 14 11:57:28.800112 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.800095    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:57:28.800168 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.800098    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:57:28.803655 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.803634    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"]
Jul 14 11:57:28.897427 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.897400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/477fdf7b-b134-4a9d-a6d2-f5ade4af2865-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9\" (UID: \"477fdf7b-b134-4a9d-a6d2-f5ade4af2865\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"
Jul 14 11:57:28.997833 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.997802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/477fdf7b-b134-4a9d-a6d2-f5ade4af2865-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9\" (UID: \"477fdf7b-b134-4a9d-a6d2-f5ade4af2865\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"
Jul 14 11:57:28.998369 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:28.998349    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/477fdf7b-b134-4a9d-a6d2-f5ade4af2865-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9\" (UID: \"477fdf7b-b134-4a9d-a6d2-f5ade4af2865\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"
Jul 14 11:57:29.110572 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:29.110487    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"
Jul 14 11:57:29.238669 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:29.238646    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"]
Jul 14 11:57:29.240846 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:57:29.240815    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477fdf7b_b134_4a9d_a6d2_f5ade4af2865.slice/crio-bbe3ab022e3f3ad7c9bbada8dbcec225225a568d5a0a47ae1f311336f7f3a10a WatchSource:0}: Error finding container bbe3ab022e3f3ad7c9bbada8dbcec225225a568d5a0a47ae1f311336f7f3a10a: Status 404 returned error can't find the container with id bbe3ab022e3f3ad7c9bbada8dbcec225225a568d5a0a47ae1f311336f7f3a10a
Jul 14 11:57:29.327641 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:29.327606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9" event={"ID":"477fdf7b-b134-4a9d-a6d2-f5ade4af2865","Type":"ContainerStarted","Data":"bbe3ab022e3f3ad7c9bbada8dbcec225225a568d5a0a47ae1f311336f7f3a10a"}
Jul 14 11:57:30.334006 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:30.333971    2564 generic.go:356] "Generic (PLEG): container finished" podID="477fdf7b-b134-4a9d-a6d2-f5ade4af2865" containerID="27e9240345baa2c6197c4d4a49bff0130f1606e6d9de6fb4bb979cbc98e74fe8" exitCode=0
Jul 14 11:57:30.334402 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:30.334059    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9" event={"ID":"477fdf7b-b134-4a9d-a6d2-f5ade4af2865","Type":"ContainerDied","Data":"27e9240345baa2c6197c4d4a49bff0130f1606e6d9de6fb4bb979cbc98e74fe8"}
Jul 14 11:57:31.467194 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:31.467164    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"
Jul 14 11:57:31.515587 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:31.515559    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/477fdf7b-b134-4a9d-a6d2-f5ade4af2865-maas-api-service-ca\") pod \"477fdf7b-b134-4a9d-a6d2-f5ade4af2865\" (UID: \"477fdf7b-b134-4a9d-a6d2-f5ade4af2865\") "
Jul 14 11:57:31.515927 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:31.515904    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/477fdf7b-b134-4a9d-a6d2-f5ade4af2865-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "477fdf7b-b134-4a9d-a6d2-f5ade4af2865" (UID: "477fdf7b-b134-4a9d-a6d2-f5ade4af2865"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:57:31.616718 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:31.616622    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/477fdf7b-b134-4a9d-a6d2-f5ade4af2865-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:57:32.345090 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:32.345046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9" event={"ID":"477fdf7b-b134-4a9d-a6d2-f5ade4af2865","Type":"ContainerDied","Data":"bbe3ab022e3f3ad7c9bbada8dbcec225225a568d5a0a47ae1f311336f7f3a10a"}
Jul 14 11:57:32.345090 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:32.345070    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9"
Jul 14 11:57:32.345090 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:32.345096    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbe3ab022e3f3ad7c9bbada8dbcec225225a568d5a0a47ae1f311336f7f3a10a"
Jul 14 11:57:42.557270 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:42.557233    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:57:54.992020 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:54.991987    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"]
Jul 14 11:57:54.992469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:54.992315    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477fdf7b-b134-4a9d-a6d2-f5ade4af2865" containerName="revoke-keys"
Jul 14 11:57:54.992469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:54.992326    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="477fdf7b-b134-4a9d-a6d2-f5ade4af2865" containerName="revoke-keys"
Jul 14 11:57:54.992469 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:54.992388    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="477fdf7b-b134-4a9d-a6d2-f5ade4af2865" containerName="revoke-keys"
Jul 14 11:57:54.995691 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:54.995668    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"
Jul 14 11:57:54.998065 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:54.998041    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 11:57:54.998182 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:54.998120    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 11:57:55.008964 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:55.008935    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"]
Jul 14 11:57:55.121088 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:55.121046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54044d7c-b933-4210-a26a-a604ff2300a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-290745-nc7ck\" (UID: \"54044d7c-b933-4210-a26a-a604ff2300a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"
Jul 14 11:57:55.222427 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:55.222391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54044d7c-b933-4210-a26a-a604ff2300a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-290745-nc7ck\" (UID: \"54044d7c-b933-4210-a26a-a604ff2300a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"
Jul 14 11:57:55.223046 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:55.223028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54044d7c-b933-4210-a26a-a604ff2300a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-290745-nc7ck\" (UID: \"54044d7c-b933-4210-a26a-a604ff2300a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"
Jul 14 11:57:55.306582 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:55.306537    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"
Jul 14 11:57:55.437497 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:55.437455    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"]
Jul 14 11:57:55.439669 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:57:55.439643    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54044d7c_b933_4210_a26a_a604ff2300a7.slice/crio-9a3f977f3eb775f2413b12ca21fdfa75e6738a728ecb006324923c784149ebfb WatchSource:0}: Error finding container 9a3f977f3eb775f2413b12ca21fdfa75e6738a728ecb006324923c784149ebfb: Status 404 returned error can't find the container with id 9a3f977f3eb775f2413b12ca21fdfa75e6738a728ecb006324923c784149ebfb
Jul 14 11:57:55.455224 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:55.455197    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck" event={"ID":"54044d7c-b933-4210-a26a-a604ff2300a7","Type":"ContainerStarted","Data":"9a3f977f3eb775f2413b12ca21fdfa75e6738a728ecb006324923c784149ebfb"}
Jul 14 11:57:56.461393 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:56.461303    2564 generic.go:356] "Generic (PLEG): container finished" podID="54044d7c-b933-4210-a26a-a604ff2300a7" containerID="a41ba081e5cc0485df6488886ab4d043811e12337d68a5305718d9da3e6d0689" exitCode=0
Jul 14 11:57:56.461781 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:56.461392    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck" event={"ID":"54044d7c-b933-4210-a26a-a604ff2300a7","Type":"ContainerDied","Data":"a41ba081e5cc0485df6488886ab4d043811e12337d68a5305718d9da3e6d0689"}
Jul 14 11:57:57.595415 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:57.595392    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"
Jul 14 11:57:57.743809 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:57.743692    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54044d7c-b933-4210-a26a-a604ff2300a7-maas-api-service-ca\") pod \"54044d7c-b933-4210-a26a-a604ff2300a7\" (UID: \"54044d7c-b933-4210-a26a-a604ff2300a7\") "
Jul 14 11:57:57.744058 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:57.744037    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54044d7c-b933-4210-a26a-a604ff2300a7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54044d7c-b933-4210-a26a-a604ff2300a7" (UID: "54044d7c-b933-4210-a26a-a604ff2300a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 11:57:57.844871 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:57.844837    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54044d7c-b933-4210-a26a-a604ff2300a7-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 11:57:58.471634 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:58.471603    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck"
Jul 14 11:57:58.471822 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:58.471602    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-290745-nc7ck" event={"ID":"54044d7c-b933-4210-a26a-a604ff2300a7","Type":"ContainerDied","Data":"9a3f977f3eb775f2413b12ca21fdfa75e6738a728ecb006324923c784149ebfb"}
Jul 14 11:57:58.471822 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:57:58.471762    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a3f977f3eb775f2413b12ca21fdfa75e6738a728ecb006324923c784149ebfb"
Jul 14 11:58:08.660900 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:08.660859    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:58:26.785468 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:26.785439    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:58:26.791522 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:26.791501    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 11:58:29.999206 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:29.999171    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:58:39.305375 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:39.305331    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:58:53.568826 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.568742    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"]
Jul 14 11:58:53.569272 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.569230    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54044d7c-b933-4210-a26a-a604ff2300a7" containerName="revoke-keys"
Jul 14 11:58:53.569272 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.569245    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54044d7c-b933-4210-a26a-a604ff2300a7" containerName="revoke-keys"
Jul 14 11:58:53.569358 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.569325    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="54044d7c-b933-4210-a26a-a604ff2300a7" containerName="revoke-keys"
Jul 14 11:58:53.572659 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.572638    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.575728 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.575680    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"openshift-service-ca.crt\""
Jul 14 11:58:53.575840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.575803    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"auth-test-model-1cc796-kserve-self-signed-certs\""
Jul 14 11:58:53.575840 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.575816    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"default-dockercfg-mgrf7\""
Jul 14 11:58:53.576190 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.576175    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"kube-root-ca.crt\""
Jul 14 11:58:53.582005 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.581982    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"]
Jul 14 11:58:53.613028 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.612998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tls-certs\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.613132 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.613040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-home\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.613132 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.613067    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx55h\" (UniqueName: \"kubernetes.io/projected/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-kube-api-access-qx55h\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.613205 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.613167    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-model-cache\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.613205 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.613197    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-dshm\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.613274 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.613238    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tmp-dir\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714303 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714274    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tls-certs\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714419 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-home\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714419 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx55h\" (UniqueName: \"kubernetes.io/projected/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-kube-api-access-qx55h\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714419 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-model-cache\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714419 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-dshm\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714555 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714428    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tmp-dir\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714783 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714761    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-home\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714844 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tmp-dir\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.714844 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.714817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-model-cache\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.716685 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.716665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-dshm\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.716857 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.716841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tls-certs\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.724351 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.724327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx55h\" (UniqueName: \"kubernetes.io/projected/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-kube-api-access-qx55h\") pod \"auth-test-model-1cc796-kserve-85fc4bbcd7-987g2\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") " pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:53.882671 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:53.882595    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:58:54.012350 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:54.012326    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"]
Jul 14 11:58:54.014271 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:58:54.014243    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d7dd6f4_cf1f_4441_8d68_d5192c5288f6.slice/crio-d39f1e638c758249cf9db37fb872a91df4a19dff7fc82adaaa9151681f9eead6 WatchSource:0}: Error finding container d39f1e638c758249cf9db37fb872a91df4a19dff7fc82adaaa9151681f9eead6: Status 404 returned error can't find the container with id d39f1e638c758249cf9db37fb872a91df4a19dff7fc82adaaa9151681f9eead6
Jul 14 11:58:54.754792 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:54.754758    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2" event={"ID":"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6","Type":"ContainerStarted","Data":"ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39"}
Jul 14 11:58:54.755134 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:54.754798    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2" event={"ID":"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6","Type":"ContainerStarted","Data":"d39f1e638c758249cf9db37fb872a91df4a19dff7fc82adaaa9151681f9eead6"}
Jul 14 11:58:54.772843 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:54.772798    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2" podStartSLOduration=1.772784695 podStartE2EDuration="1.772784695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:58:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:58:54.770935821 +0000 UTC m=+2428.571491561" watchObservedRunningTime="2026-07-14 11:58:54.772784695 +0000 UTC m=+2428.573340435"
Jul 14 11:58:56.770738 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:58:56.770709    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:59:03.883349 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:03.883307    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:59:03.883349 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:03.883355    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:59:03.895925 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:03.895897    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:59:04.813307 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:04.813271    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 11:59:25.184665 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.184631    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"]
Jul 14 11:59:25.188516 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.188498    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.189913 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.189891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tmp-dir\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.190014 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.189925    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-model-cache\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.190014 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.189957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-home\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.190014 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.189971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-dshm\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.190014 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.190001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb5nn\" (UniqueName: \"kubernetes.io/projected/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-kube-api-access-vb5nn\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.190151 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.190019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tls-certs\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.192178 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.192119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"auth-test-model-0508a3-kserve-self-signed-certs\""
Jul 14 11:59:25.192178 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.192119    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"kube-root-ca.crt\""
Jul 14 11:59:25.192178 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.192173    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"default-dockercfg-ks24p\""
Jul 14 11:59:25.192390 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.192173    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"openshift-service-ca.crt\""
Jul 14 11:59:25.198431 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.198404    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"]
Jul 14 11:59:25.290592 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.290555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb5nn\" (UniqueName: \"kubernetes.io/projected/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-kube-api-access-vb5nn\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.290592 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.290592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tls-certs\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.290890 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.290644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tmp-dir\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.290890 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.290664    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-model-cache\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.290890 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.290715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-home\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.290890 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.290735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-dshm\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.291169 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.291142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-model-cache\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.291241 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.291209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-home\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.291241 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.291232    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tmp-dir\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.293009 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.292992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-dshm\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.293263 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.293244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tls-certs\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.299262 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.299241    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb5nn\" (UniqueName: \"kubernetes.io/projected/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-kube-api-access-vb5nn\") pod \"auth-test-model-0508a3-kserve-6745785899-t9klw\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") " pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.499971 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.499881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:25.631073 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.631045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"]
Jul 14 11:59:25.632378 ip-10-0-142-172 kubenswrapper[2564]: W0714 11:59:25.632352    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbd9b4b2_dfa1_4c6e_af0b_d956ca46a791.slice/crio-605f64f0aab72bea43992bc4c094400d25be812b865276828964abd0c25fa576 WatchSource:0}: Error finding container 605f64f0aab72bea43992bc4c094400d25be812b865276828964abd0c25fa576: Status 404 returned error can't find the container with id 605f64f0aab72bea43992bc4c094400d25be812b865276828964abd0c25fa576
Jul 14 11:59:25.907300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.907265    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw" event={"ID":"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791","Type":"ContainerStarted","Data":"4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb"}
Jul 14 11:59:25.907300 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.907308    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw" event={"ID":"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791","Type":"ContainerStarted","Data":"605f64f0aab72bea43992bc4c094400d25be812b865276828964abd0c25fa576"}
Jul 14 11:59:25.929204 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:25.929157    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw" podStartSLOduration=0.92914092 podStartE2EDuration="929.14092ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:59:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:59:25.928300091 +0000 UTC m=+2459.728855833" watchObservedRunningTime="2026-07-14 11:59:25.92914092 +0000 UTC m=+2459.729696714"
Jul 14 11:59:29.372753 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:29.372691    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 11:59:35.500891 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:35.500845    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:35.501532 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:35.500914    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:35.513446 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:35.513413    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:35.967737 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:35.967692    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 11:59:44.265777 ip-10-0-142-172 kubenswrapper[2564]: I0714 11:59:44.265746    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:00:00.144337 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:00.144299    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"]
Jul 14 12:00:00.147328 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:00.147306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"
Jul 14 12:00:00.149614 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:00.149597    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 12:00:00.156192 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:00.156171    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"]
Jul 14 12:00:00.158165 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:00.158148    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"
Jul 14 12:00:00.289600 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:00.289574    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"]
Jul 14 12:00:00.291393 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:00:00.291364    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb65ac07_074f_4ed1_a0c4_325550d72ea2.slice/crio-4ba0bc28ba6c4348c7db72f2a14df34349d971dca1ac05bd73afc507f10941e4 WatchSource:0}: Error finding container 4ba0bc28ba6c4348c7db72f2a14df34349d971dca1ac05bd73afc507f10941e4: Status 404 returned error can't find the container with id 4ba0bc28ba6c4348c7db72f2a14df34349d971dca1ac05bd73afc507f10941e4
Jul 14 12:00:01.089918 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:01.089887    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv" event={"ID":"eb65ac07-074f-4ed1-a0c4-325550d72ea2","Type":"ContainerStarted","Data":"c40ff555295be7164bd521dbf3d7060a5ccbfe8acd9d329da4048320f6d2aa01"}
Jul 14 12:00:01.090058 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:01.089938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv" event={"ID":"eb65ac07-074f-4ed1-a0c4-325550d72ea2","Type":"ContainerStarted","Data":"4ba0bc28ba6c4348c7db72f2a14df34349d971dca1ac05bd73afc507f10941e4"}
Jul 14 12:00:01.107750 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:01.107684    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv" podStartSLOduration=1.107670176 podStartE2EDuration="1.107670176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:00:01.105854551 +0000 UTC m=+2494.906410318" watchObservedRunningTime="2026-07-14 12:00:01.107670176 +0000 UTC m=+2494.908225916"
Jul 14 12:00:02.097331 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:02.097291    2564 generic.go:356] "Generic (PLEG): container finished" podID="eb65ac07-074f-4ed1-a0c4-325550d72ea2" containerID="c40ff555295be7164bd521dbf3d7060a5ccbfe8acd9d329da4048320f6d2aa01" exitCode=0
Jul 14 12:00:02.097725 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:02.097359    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv" event={"ID":"eb65ac07-074f-4ed1-a0c4-325550d72ea2","Type":"ContainerDied","Data":"c40ff555295be7164bd521dbf3d7060a5ccbfe8acd9d329da4048320f6d2aa01"}
Jul 14 12:00:03.230821 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:03.230796    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"
Jul 14 12:00:03.264493 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:03.264458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:00:04.108875 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:04.108843    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"
Jul 14 12:00:04.109058 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:04.108877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv" event={"ID":"eb65ac07-074f-4ed1-a0c4-325550d72ea2","Type":"ContainerDied","Data":"4ba0bc28ba6c4348c7db72f2a14df34349d971dca1ac05bd73afc507f10941e4"}
Jul 14 12:00:04.109058 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:04.108927    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ba0bc28ba6c4348c7db72f2a14df34349d971dca1ac05bd73afc507f10941e4"
Jul 14 12:00:17.964892 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:17.964853    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:00:32.364488 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:32.364453    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:00:44.963767 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:00:44.963732    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:01:01.769404 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:01.769368    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:01:15.927223 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:15.927180    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"]
Jul 14 12:01:16.559987 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:16.559953    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"]
Jul 14 12:01:20.471423 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:20.471384    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:01:30.947707 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:30.947654    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2" podUID="8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" containerName="main" containerID="cri-o://ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39" gracePeriod=45
Jul 14 12:01:31.198304 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.198246    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 12:01:31.266021 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.265985    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-home\") pod \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") "
Jul 14 12:01:31.266195 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266030    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx55h\" (UniqueName: \"kubernetes.io/projected/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-kube-api-access-qx55h\") pod \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") "
Jul 14 12:01:31.266195 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266057    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tmp-dir\") pod \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") "
Jul 14 12:01:31.266195 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266090    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-dshm\") pod \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") "
Jul 14 12:01:31.266195 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266125    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tls-certs\") pod \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") "
Jul 14 12:01:31.266411 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266207    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-model-cache\") pod \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\" (UID: \"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6\") "
Jul 14 12:01:31.266411 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266283    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-home" (OuterVolumeSpecName: "home") pod "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" (UID: "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.266411 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266327    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" (UID: "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.266563 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266504    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" (UID: "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.266563 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266556    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.266643 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.266570    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.268360 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.268319    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-kube-api-access-qx55h" (OuterVolumeSpecName: "kube-api-access-qx55h") pod "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" (UID: "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6"). InnerVolumeSpecName "kube-api-access-qx55h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:01:31.268483 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.268385    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" (UID: "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 12:01:31.268483 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.268392    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-dshm" (OuterVolumeSpecName: "dshm") pod "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" (UID: "8d7dd6f4-cf1f-4441-8d68-d5192c5288f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.367763 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.367736    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.367763 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.367757    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qx55h\" (UniqueName: \"kubernetes.io/projected/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-kube-api-access-qx55h\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.367763 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.367766    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.367967 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.367775    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.544413 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.544375    2564 generic.go:356] "Generic (PLEG): container finished" podID="8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" containerID="ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39" exitCode=0
Jul 14 12:01:31.544585 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.544434    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"
Jul 14 12:01:31.544585 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.544454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2" event={"ID":"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6","Type":"ContainerDied","Data":"ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39"}
Jul 14 12:01:31.544585 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.544494    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2" event={"ID":"8d7dd6f4-cf1f-4441-8d68-d5192c5288f6","Type":"ContainerDied","Data":"d39f1e638c758249cf9db37fb872a91df4a19dff7fc82adaaa9151681f9eead6"}
Jul 14 12:01:31.544585 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.544517    2564 scope.go:117] "RemoveContainer" containerID="ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39"
Jul 14 12:01:31.554036 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.554016    2564 scope.go:117] "RemoveContainer" containerID="ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39"
Jul 14 12:01:31.554277 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:01:31.554258    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39\": container with ID starting with ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39 not found: ID does not exist" containerID="ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39"
Jul 14 12:01:31.554316 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.554285    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39"} err="failed to get container status \"ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39\": rpc error: code = NotFound desc = could not find container \"ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39\": container with ID starting with ff5de2fb9d70a29886de169a1597f1f7003a9b083af7946a9ecf6f24eccedc39 not found: ID does not exist"
Jul 14 12:01:31.568470 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.568441    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"]
Jul 14 12:01:31.572589 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.572564    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/auth-test-model-1cc796-kserve-85fc4bbcd7-987g2"]
Jul 14 12:01:31.579920 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.579872    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw" podUID="cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" containerName="main" containerID="cri-o://4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb" gracePeriod=45
Jul 14 12:01:31.823799 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.823773    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 12:01:31.872045 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872012    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tls-certs\") pod \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") "
Jul 14 12:01:31.872045 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872060    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb5nn\" (UniqueName: \"kubernetes.io/projected/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-kube-api-access-vb5nn\") pod \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") "
Jul 14 12:01:31.872314 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872095    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-model-cache\") pod \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") "
Jul 14 12:01:31.872314 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872122    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tmp-dir\") pod \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") "
Jul 14 12:01:31.872314 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872188    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-dshm\") pod \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") "
Jul 14 12:01:31.872314 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872239    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-home\") pod \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\" (UID: \"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791\") "
Jul 14 12:01:31.872522 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872405    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-model-cache" (OuterVolumeSpecName: "model-cache") pod "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" (UID: "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.872522 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872419    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" (UID: "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.872634 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872554    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.872634 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872570    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.872634 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.872564    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-home" (OuterVolumeSpecName: "home") pod "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" (UID: "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.874302 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.874272    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-dshm" (OuterVolumeSpecName: "dshm") pod "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" (UID: "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:01:31.874423 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.874313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" (UID: "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 12:01:31.874423 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.874350    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-kube-api-access-vb5nn" (OuterVolumeSpecName: "kube-api-access-vb5nn") pod "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" (UID: "cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791"). InnerVolumeSpecName "kube-api-access-vb5nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:01:31.973665 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.973633    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.973665 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.973659    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.974034 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.973671    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb5nn\" (UniqueName: \"kubernetes.io/projected/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-kube-api-access-vb5nn\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:31.974034 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:31.973679    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:01:32.552032 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.551996    2564 generic.go:356] "Generic (PLEG): container finished" podID="cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" containerID="4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb" exitCode=0
Jul 14 12:01:32.552032 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.552031    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"
Jul 14 12:01:32.552286 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.552082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw" event={"ID":"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791","Type":"ContainerDied","Data":"4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb"}
Jul 14 12:01:32.552286 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.552120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw" event={"ID":"cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791","Type":"ContainerDied","Data":"605f64f0aab72bea43992bc4c094400d25be812b865276828964abd0c25fa576"}
Jul 14 12:01:32.552286 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.552141    2564 scope.go:117] "RemoveContainer" containerID="4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb"
Jul 14 12:01:32.563944 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.563922    2564 scope.go:117] "RemoveContainer" containerID="4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb"
Jul 14 12:01:32.564191 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:01:32.564172    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb\": container with ID starting with 4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb not found: ID does not exist" containerID="4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb"
Jul 14 12:01:32.564253 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.564200    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb"} err="failed to get container status \"4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb\": rpc error: code = NotFound desc = could not find container \"4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb\": container with ID starting with 4fe8d29a0484760562095c06fb574359b7ff9e99c4af251b914aa5f8476954fb not found: ID does not exist"
Jul 14 12:01:32.574442 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.574420    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"]
Jul 14 12:01:32.581320 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.581297    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/auth-test-model-0508a3-kserve-6745785899-t9klw"]
Jul 14 12:01:32.769403 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.769368    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" path="/var/lib/kubelet/pods/8d7dd6f4-cf1f-4441-8d68-d5192c5288f6/volumes"
Jul 14 12:01:32.769773 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:32.769759    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" path="/var/lib/kubelet/pods/cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791/volumes"
Jul 14 12:01:34.878671 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.878636    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"]
Jul 14 12:01:34.879143 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879126    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" containerName="main"
Jul 14 12:01:34.879200 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879146    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" containerName="main"
Jul 14 12:01:34.879200 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879160    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb65ac07-074f-4ed1-a0c4-325550d72ea2" containerName="cleanup"
Jul 14 12:01:34.879200 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879170    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb65ac07-074f-4ed1-a0c4-325550d72ea2" containerName="cleanup"
Jul 14 12:01:34.879200 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879196    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" containerName="main"
Jul 14 12:01:34.879329 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879205    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" containerName="main"
Jul 14 12:01:34.879329 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879315    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd9b4b2-dfa1-4c6e-af0b-d956ca46a791" containerName="main"
Jul 14 12:01:34.879393 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879330    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d7dd6f4-cf1f-4441-8d68-d5192c5288f6" containerName="main"
Jul 14 12:01:34.879393 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.879339    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb65ac07-074f-4ed1-a0c4-325550d72ea2" containerName="cleanup"
Jul 14 12:01:34.882658 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.882638    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:34.884958 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.884936    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"kube-root-ca.crt\""
Jul 14 12:01:34.885663 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.885643    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"sub-test-model-1cc796-kserve-self-signed-certs\""
Jul 14 12:01:34.885753 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.885682    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"openshift-service-ca.crt\""
Jul 14 12:01:34.885812 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.885766    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"default-dockercfg-mgrf7\""
Jul 14 12:01:34.893066 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:34.893044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"]
Jul 14 12:01:35.000401 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.000358    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tmp-dir\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.000401 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.000400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-dshm\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.000639 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.000425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tls-certs\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.000639 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.000499    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fszj7\" (UniqueName: \"kubernetes.io/projected/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-kube-api-access-fszj7\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.000639 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.000572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-model-cache\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.000639 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.000593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-home\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.101658 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.101622    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tls-certs\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.101848 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.101669    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fszj7\" (UniqueName: \"kubernetes.io/projected/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-kube-api-access-fszj7\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.101848 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.101747    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-model-cache\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.101848 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.101768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-home\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.101848 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.101812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tmp-dir\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.101848 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.101842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-dshm\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.102228 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.102208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-model-cache\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.102316 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.102255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-home\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.102316 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.102267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tmp-dir\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.103989 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.103966    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-dshm\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.104155 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.104138    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tls-certs\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.111086 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.111066    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fszj7\" (UniqueName: \"kubernetes.io/projected/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-kube-api-access-fszj7\") pod \"sub-test-model-1cc796-kserve-65fbfb9657-zt74w\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") " pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.192925 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.192841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:35.322634 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.322605    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"]
Jul 14 12:01:35.324184 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:01:35.324152    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebc21f1e_26e3_431d_8c5f_ebfc0e9ee00e.slice/crio-ad3b3a02ba8ceecf3e251ee64b86a3f69dbceb27fc2acab68ab1928b4f465894 WatchSource:0}: Error finding container ad3b3a02ba8ceecf3e251ee64b86a3f69dbceb27fc2acab68ab1928b4f465894: Status 404 returned error can't find the container with id ad3b3a02ba8ceecf3e251ee64b86a3f69dbceb27fc2acab68ab1928b4f465894
Jul 14 12:01:35.568529 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.568489    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w" event={"ID":"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e","Type":"ContainerStarted","Data":"fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712"}
Jul 14 12:01:35.568529 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.568531    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w" event={"ID":"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e","Type":"ContainerStarted","Data":"ad3b3a02ba8ceecf3e251ee64b86a3f69dbceb27fc2acab68ab1928b4f465894"}
Jul 14 12:01:35.587380 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:35.587331    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w" podStartSLOduration=1.587313894 podStartE2EDuration="1.587313894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:01:35.586238148 +0000 UTC m=+2589.386793899" watchObservedRunningTime="2026-07-14 12:01:35.587313894 +0000 UTC m=+2589.387869633"
Jul 14 12:01:38.587597 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:38.587565    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:01:45.193461 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:45.193426    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:45.193961 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:45.193475    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:45.205825 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:45.205798    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:01:45.651386 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:01:45.651353    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:02:05.974510 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:05.974476    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"]
Jul 14 12:02:05.977832 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:05.977811    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:05.980260 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:05.980237    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"default-dockercfg-ks24p\""
Jul 14 12:02:05.980914 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:05.980891    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"kube-root-ca.crt\""
Jul 14 12:02:05.981028 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:05.980893    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"sub-test-model-0508a3-kserve-self-signed-certs\""
Jul 14 12:02:05.981028 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:05.980936    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"openshift-service-ca.crt\""
Jul 14 12:02:05.990589 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:05.990565    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"]
Jul 14 12:02:06.066589 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.066554    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-model-cache\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.066821 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.066600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-dshm\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.066821 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.066626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bebba93-2967-43c4-9897-15f1cf7865d7-tls-certs\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.066821 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.066685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-home\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.066821 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.066741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-tmp-dir\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.066821 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.066792    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brsqm\" (UniqueName: \"kubernetes.io/projected/8bebba93-2967-43c4-9897-15f1cf7865d7-kube-api-access-brsqm\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.167649 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.167613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-model-cache\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.167877 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.167661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-dshm\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.167877 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.167719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bebba93-2967-43c4-9897-15f1cf7865d7-tls-certs\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.167877 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.167760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-home\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.167877 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.167780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-tmp-dir\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.167877 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.167838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brsqm\" (UniqueName: \"kubernetes.io/projected/8bebba93-2967-43c4-9897-15f1cf7865d7-kube-api-access-brsqm\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.168122 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.168098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-model-cache\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.168176 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.168157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-home\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.168281 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.168263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-tmp-dir\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.170065 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.170040    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-dshm\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.170294 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.170279    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bebba93-2967-43c4-9897-15f1cf7865d7-tls-certs\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.176096 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.176071    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brsqm\" (UniqueName: \"kubernetes.io/projected/8bebba93-2967-43c4-9897-15f1cf7865d7-kube-api-access-brsqm\") pod \"sub-test-model-0508a3-kserve-8c9ff967c-2clhz\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") " pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.287144 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.287114    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:06.417876 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.417834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"]
Jul 14 12:02:06.421410 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:02:06.421370    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bebba93_2967_43c4_9897_15f1cf7865d7.slice/crio-986e5e4ca5677e551764b3965d1824cc6578160b98eb799c9574f13a61bf157f WatchSource:0}: Error finding container 986e5e4ca5677e551764b3965d1824cc6578160b98eb799c9574f13a61bf157f: Status 404 returned error can't find the container with id 986e5e4ca5677e551764b3965d1824cc6578160b98eb799c9574f13a61bf157f
Jul 14 12:02:06.739485 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.739380    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz" event={"ID":"8bebba93-2967-43c4-9897-15f1cf7865d7","Type":"ContainerStarted","Data":"8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a"}
Jul 14 12:02:06.739485 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.739434    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz" event={"ID":"8bebba93-2967-43c4-9897-15f1cf7865d7","Type":"ContainerStarted","Data":"986e5e4ca5677e551764b3965d1824cc6578160b98eb799c9574f13a61bf157f"}
Jul 14 12:02:06.762247 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:06.762183    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz" podStartSLOduration=1.762165013 podStartE2EDuration="1.762165013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:02:06.76007255 +0000 UTC m=+2620.560628295" watchObservedRunningTime="2026-07-14 12:02:06.762165013 +0000 UTC m=+2620.562720752"
Jul 14 12:02:11.169442 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:11.169402    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:02:16.287319 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:16.287279    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:16.287319 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:16.287326    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:16.299860 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:16.299829    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:16.800978 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:16.800948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:25.274795 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:25.274759    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:02:42.175035 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:42.175001    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"]
Jul 14 12:02:43.159959 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:43.159929    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"]
Jul 14 12:02:46.566310 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:46.566272    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:02:57.195056 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.194991    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w" podUID="ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" containerName="main" containerID="cri-o://fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712" gracePeriod=45
Jul 14 12:02:57.435592 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.435568    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:02:57.532947 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.532923    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tls-certs\") pod \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") "
Jul 14 12:02:57.533103 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.532960    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-model-cache\") pod \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") "
Jul 14 12:02:57.533103 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533021    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tmp-dir\") pod \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") "
Jul 14 12:02:57.533205 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533113    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-home\") pod \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") "
Jul 14 12:02:57.533205 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533159    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fszj7\" (UniqueName: \"kubernetes.io/projected/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-kube-api-access-fszj7\") pod \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") "
Jul 14 12:02:57.533288 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533230    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-dshm\") pod \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\" (UID: \"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e\") "
Jul 14 12:02:57.533288 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533253    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" (UID: "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:57.533288 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533258    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" (UID: "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:57.533452 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533356    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-home" (OuterVolumeSpecName: "home") pod "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" (UID: "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:57.533511 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533486    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:57.533511 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533498    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:57.533511 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.533506    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:57.535182 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.535144    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-kube-api-access-fszj7" (OuterVolumeSpecName: "kube-api-access-fszj7") pod "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" (UID: "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e"). InnerVolumeSpecName "kube-api-access-fszj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:02:57.535182 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.535165    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" (UID: "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 12:02:57.535329 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.535198    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-dshm" (OuterVolumeSpecName: "dshm") pod "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" (UID: "ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:57.634788 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.634755    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fszj7\" (UniqueName: \"kubernetes.io/projected/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-kube-api-access-fszj7\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:57.634788 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.634786    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:57.634788 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.634797    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:57.996302 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.996202    2564 generic.go:356] "Generic (PLEG): container finished" podID="ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" containerID="fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712" exitCode=0
Jul 14 12:02:57.996302 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.996250    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"
Jul 14 12:02:57.996302 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.996286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w" event={"ID":"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e","Type":"ContainerDied","Data":"fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712"}
Jul 14 12:02:57.996569 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.996341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w" event={"ID":"ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e","Type":"ContainerDied","Data":"ad3b3a02ba8ceecf3e251ee64b86a3f69dbceb27fc2acab68ab1928b4f465894"}
Jul 14 12:02:57.996569 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:57.996361    2564 scope.go:117] "RemoveContainer" containerID="fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712"
Jul 14 12:02:58.007076 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.007060    2564 scope.go:117] "RemoveContainer" containerID="fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712"
Jul 14 12:02:58.007319 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:02:58.007300    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712\": container with ID starting with fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712 not found: ID does not exist" containerID="fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712"
Jul 14 12:02:58.007367 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.007326    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712"} err="failed to get container status \"fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712\": rpc error: code = NotFound desc = could not find container \"fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712\": container with ID starting with fe97a6f5e19dd41bbb9d107bbfe7f0cca6f7e94ce3a4e5bba12fc3b174657712 not found: ID does not exist"
Jul 14 12:02:58.019407 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.019383    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"]
Jul 14 12:02:58.023535 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.023509    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/sub-test-model-1cc796-kserve-65fbfb9657-zt74w"]
Jul 14 12:02:58.179485 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.179439    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz" podUID="8bebba93-2967-43c4-9897-15f1cf7865d7" containerName="main" containerID="cri-o://8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a" gracePeriod=45
Jul 14 12:02:58.424979 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.424957    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:58.441450 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441392    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-tmp-dir\") pod \"8bebba93-2967-43c4-9897-15f1cf7865d7\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") "
Jul 14 12:02:58.441450 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441426    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brsqm\" (UniqueName: \"kubernetes.io/projected/8bebba93-2967-43c4-9897-15f1cf7865d7-kube-api-access-brsqm\") pod \"8bebba93-2967-43c4-9897-15f1cf7865d7\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") "
Jul 14 12:02:58.441450 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441461    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-dshm\") pod \"8bebba93-2967-43c4-9897-15f1cf7865d7\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") "
Jul 14 12:02:58.441760 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441531    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-model-cache\") pod \"8bebba93-2967-43c4-9897-15f1cf7865d7\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") "
Jul 14 12:02:58.441760 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441570    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bebba93-2967-43c4-9897-15f1cf7865d7-tls-certs\") pod \"8bebba93-2967-43c4-9897-15f1cf7865d7\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") "
Jul 14 12:02:58.441760 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441616    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-home\") pod \"8bebba93-2967-43c4-9897-15f1cf7865d7\" (UID: \"8bebba93-2967-43c4-9897-15f1cf7865d7\") "
Jul 14 12:02:58.441760 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441678    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8bebba93-2967-43c4-9897-15f1cf7865d7" (UID: "8bebba93-2967-43c4-9897-15f1cf7865d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:58.441950 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.441929    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:58.442337 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.442135    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "8bebba93-2967-43c4-9897-15f1cf7865d7" (UID: "8bebba93-2967-43c4-9897-15f1cf7865d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:58.442502 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.442467    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-home" (OuterVolumeSpecName: "home") pod "8bebba93-2967-43c4-9897-15f1cf7865d7" (UID: "8bebba93-2967-43c4-9897-15f1cf7865d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:58.443926 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.443900    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bebba93-2967-43c4-9897-15f1cf7865d7-kube-api-access-brsqm" (OuterVolumeSpecName: "kube-api-access-brsqm") pod "8bebba93-2967-43c4-9897-15f1cf7865d7" (UID: "8bebba93-2967-43c4-9897-15f1cf7865d7"). InnerVolumeSpecName "kube-api-access-brsqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:02:58.444039 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.443960    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-dshm" (OuterVolumeSpecName: "dshm") pod "8bebba93-2967-43c4-9897-15f1cf7865d7" (UID: "8bebba93-2967-43c4-9897-15f1cf7865d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:02:58.444357 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.444328    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bebba93-2967-43c4-9897-15f1cf7865d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8bebba93-2967-43c4-9897-15f1cf7865d7" (UID: "8bebba93-2967-43c4-9897-15f1cf7865d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 12:02:58.543177 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.543104    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:58.543177 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.543128    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:58.543177 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.543138    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8bebba93-2967-43c4-9897-15f1cf7865d7-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:58.543177 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.543147    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8bebba93-2967-43c4-9897-15f1cf7865d7-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:58.543177 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.543155    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brsqm\" (UniqueName: \"kubernetes.io/projected/8bebba93-2967-43c4-9897-15f1cf7865d7-kube-api-access-brsqm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:02:58.770211 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:58.770180    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" path="/var/lib/kubelet/pods/ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e/volumes"
Jul 14 12:02:59.001998 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.001904    2564 generic.go:356] "Generic (PLEG): container finished" podID="8bebba93-2967-43c4-9897-15f1cf7865d7" containerID="8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a" exitCode=0
Jul 14 12:02:59.001998 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.001944    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"
Jul 14 12:02:59.001998 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.001989    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz" event={"ID":"8bebba93-2967-43c4-9897-15f1cf7865d7","Type":"ContainerDied","Data":"8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a"}
Jul 14 12:02:59.002247 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.002031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz" event={"ID":"8bebba93-2967-43c4-9897-15f1cf7865d7","Type":"ContainerDied","Data":"986e5e4ca5677e551764b3965d1824cc6578160b98eb799c9574f13a61bf157f"}
Jul 14 12:02:59.002247 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.002047    2564 scope.go:117] "RemoveContainer" containerID="8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a"
Jul 14 12:02:59.010462 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.010444    2564 scope.go:117] "RemoveContainer" containerID="8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a"
Jul 14 12:02:59.010747 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:02:59.010716    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a\": container with ID starting with 8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a not found: ID does not exist" containerID="8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a"
Jul 14 12:02:59.010815 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.010758    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a"} err="failed to get container status \"8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a\": rpc error: code = NotFound desc = could not find container \"8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a\": container with ID starting with 8555d72209a5bdaddecfe9a7b27dc02c30e30f0215243df41e19af6e760cc07a not found: ID does not exist"
Jul 14 12:02:59.017259 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.017234    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"]
Jul 14 12:02:59.021451 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:02:59.021431    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/sub-test-model-0508a3-kserve-8c9ff967c-2clhz"]
Jul 14 12:03:00.252382 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.252347    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"]
Jul 14 12:03:00.252761 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.252716    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" containerName="main"
Jul 14 12:03:00.252761 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.252727    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" containerName="main"
Jul 14 12:03:00.252761 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.252735    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bebba93-2967-43c4-9897-15f1cf7865d7" containerName="main"
Jul 14 12:03:00.252761 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.252741    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bebba93-2967-43c4-9897-15f1cf7865d7" containerName="main"
Jul 14 12:03:00.252906 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.252814    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bebba93-2967-43c4-9897-15f1cf7865d7" containerName="main"
Jul 14 12:03:00.252906 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.252825    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebc21f1e-26e3-431d-8c5f-ebfc0e9ee00e" containerName="main"
Jul 14 12:03:00.257009 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.256988    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.259234 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.259215    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"kube-root-ca.crt\""
Jul 14 12:03:00.259876 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.259856    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"rate-test-model-1cc796-kserve-self-signed-certs\""
Jul 14 12:03:00.259971 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.259871    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"default-dockercfg-mgrf7\""
Jul 14 12:03:00.259971 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.259939    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1cc796\"/\"openshift-service-ca.crt\""
Jul 14 12:03:00.265822 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.265802    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"]
Jul 14 12:03:00.355683 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.355638    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-dshm\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.355981 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.355961    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-model-cache\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.356144 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.356130    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-home\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.356277 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.356264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tmp-dir\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457091 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-dshm\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457091 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457094    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tls-certs\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457356 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457126    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-model-cache\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457356 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-home\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457356 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457255    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tmp-dir\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457356 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457285    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctv2h\" (UniqueName: \"kubernetes.io/projected/d9fa83bd-4b26-4c8d-8602-e966b1324e95-kube-api-access-ctv2h\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457615 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457595    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-model-cache\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457659 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tmp-dir\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.457659 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.457652    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-home\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.459362 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.459342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-dshm\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.558388 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.558350    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctv2h\" (UniqueName: \"kubernetes.io/projected/d9fa83bd-4b26-4c8d-8602-e966b1324e95-kube-api-access-ctv2h\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.558549 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.558434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tls-certs\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.560807 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.560779    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tls-certs\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.566383 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.566358    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctv2h\" (UniqueName: \"kubernetes.io/projected/d9fa83bd-4b26-4c8d-8602-e966b1324e95-kube-api-access-ctv2h\") pod \"rate-test-model-1cc796-kserve-d994d856c-bzbcn\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") " pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.566501 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.566491    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:00.698110 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.698085    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"]
Jul 14 12:03:00.700010 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:03:00.699982    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9fa83bd_4b26_4c8d_8602_e966b1324e95.slice/crio-41fc62e042d65fbb902ddf272f90dafcf9ac05e4c7ee25c56b47eb52f8395560 WatchSource:0}: Error finding container 41fc62e042d65fbb902ddf272f90dafcf9ac05e4c7ee25c56b47eb52f8395560: Status 404 returned error can't find the container with id 41fc62e042d65fbb902ddf272f90dafcf9ac05e4c7ee25c56b47eb52f8395560
Jul 14 12:03:00.769455 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:00.769425    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bebba93-2967-43c4-9897-15f1cf7865d7" path="/var/lib/kubelet/pods/8bebba93-2967-43c4-9897-15f1cf7865d7/volumes"
Jul 14 12:03:01.015508 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:01.015423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn" event={"ID":"d9fa83bd-4b26-4c8d-8602-e966b1324e95","Type":"ContainerStarted","Data":"5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7"}
Jul 14 12:03:01.015508 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:01.015465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn" event={"ID":"d9fa83bd-4b26-4c8d-8602-e966b1324e95","Type":"ContainerStarted","Data":"41fc62e042d65fbb902ddf272f90dafcf9ac05e4c7ee25c56b47eb52f8395560"}
Jul 14 12:03:01.033558 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:01.033509    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn" podStartSLOduration=1.033495659 podStartE2EDuration="1.033495659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:03:01.032398067 +0000 UTC m=+2674.832953810" watchObservedRunningTime="2026-07-14 12:03:01.033495659 +0000 UTC m=+2674.834051390"
Jul 14 12:03:05.568065 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:05.568026    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:03:10.567289 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:10.567246    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:10.567289 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:10.567296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:10.579841 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:10.579812    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:11.077828 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:11.077801    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:03:26.814707 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:26.814663    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 12:03:26.821009 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:26.820989    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d5lm4_442f870e-3885-472d-8215-b2a5eeaa9a3a/ovn-acl-logging/0.log"
Jul 14 12:03:27.476393 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.476361    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"]
Jul 14 12:03:27.481303 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.481283    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.483633 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.483612    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"kube-root-ca.crt\""
Jul 14 12:03:27.483765 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.483617    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"openshift-service-ca.crt\""
Jul 14 12:03:27.484069 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.484056    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"default-dockercfg-ks24p\""
Jul 14 12:03:27.484118 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.484082    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0508a3\"/\"rate-test-model-0508a3-kserve-self-signed-certs\""
Jul 14 12:03:27.491142 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.491120    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"]
Jul 14 12:03:27.589344 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.589306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be5e175d-b84e-4701-9772-2c3659cdf10d-tls-certs\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.589526 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.589357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-tmp-dir\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.589526 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.589380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-home\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.589526 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.589455    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-dshm\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.589526 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.589508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-model-cache\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.589666 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.589596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh9b7\" (UniqueName: \"kubernetes.io/projected/be5e175d-b84e-4701-9772-2c3659cdf10d-kube-api-access-hh9b7\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690406 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh9b7\" (UniqueName: \"kubernetes.io/projected/be5e175d-b84e-4701-9772-2c3659cdf10d-kube-api-access-hh9b7\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690406 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be5e175d-b84e-4701-9772-2c3659cdf10d-tls-certs\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690629 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-tmp-dir\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690629 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-home\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690629 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-dshm\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690773 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-model-cache\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690979 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690956    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-tmp-dir\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.690979 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.690973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-home\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.691115 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.691035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-model-cache\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.692838 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.692818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-dshm\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.693098 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.693081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be5e175d-b84e-4701-9772-2c3659cdf10d-tls-certs\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.698716 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.698677    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh9b7\" (UniqueName: \"kubernetes.io/projected/be5e175d-b84e-4701-9772-2c3659cdf10d-kube-api-access-hh9b7\") pod \"rate-test-model-0508a3-kserve-56c5bd8687-k9dbq\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") " pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.791449 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.791417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:27.925586 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:27.925558    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"]
Jul 14 12:03:27.927707 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:03:27.927663    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe5e175d_b84e_4701_9772_2c3659cdf10d.slice/crio-ef74eb42a880eac70630c3eb41fe61542dbf0d42594c89f6b1f435b5fe46faf6 WatchSource:0}: Error finding container ef74eb42a880eac70630c3eb41fe61542dbf0d42594c89f6b1f435b5fe46faf6: Status 404 returned error can't find the container with id ef74eb42a880eac70630c3eb41fe61542dbf0d42594c89f6b1f435b5fe46faf6
Jul 14 12:03:28.162065 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:28.161966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq" event={"ID":"be5e175d-b84e-4701-9772-2c3659cdf10d","Type":"ContainerStarted","Data":"e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930"}
Jul 14 12:03:28.162065 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:28.162013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq" event={"ID":"be5e175d-b84e-4701-9772-2c3659cdf10d","Type":"ContainerStarted","Data":"ef74eb42a880eac70630c3eb41fe61542dbf0d42594c89f6b1f435b5fe46faf6"}
Jul 14 12:03:28.182282 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:28.182228    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq" podStartSLOduration=1.182212791 podStartE2EDuration="1.182212791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:03:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:03:28.180921851 +0000 UTC m=+2701.981477591" watchObservedRunningTime="2026-07-14 12:03:28.182212791 +0000 UTC m=+2701.982768560"
Jul 14 12:03:31.470811 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:31.470775    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:03:37.792172 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:37.792138    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:37.792538 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:37.792197    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:37.804912 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:37.804885    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:03:38.230388 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:03:38.230300    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:04:01.059547 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:01.059509    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:04:16.164831 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:16.164789    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:04:23.108665 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:23.108630    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"]
Jul 14 12:04:24.077273 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:24.077235    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"]
Jul 14 12:04:27.813045 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:27.813010    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7264-140675705897648"]
Jul 14 12:04:27.816518 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:27.816496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705897648"
Jul 14 12:04:27.823158 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:27.823126    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-dn6f4\""
Jul 14 12:04:27.833752 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:27.833667    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705897648"]
Jul 14 12:04:27.917839 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:27.917806    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfkjr\" (UniqueName: \"kubernetes.io/projected/d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8-kube-api-access-sfkjr\") pod \"test-curl-7264-140675705897648\" (UID: \"d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8\") " pod="openshift-ingress/test-curl-7264-140675705897648"
Jul 14 12:04:28.018572 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:28.018535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfkjr\" (UniqueName: \"kubernetes.io/projected/d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8-kube-api-access-sfkjr\") pod \"test-curl-7264-140675705897648\" (UID: \"d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8\") " pod="openshift-ingress/test-curl-7264-140675705897648"
Jul 14 12:04:28.027623 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:28.027598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfkjr\" (UniqueName: \"kubernetes.io/projected/d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8-kube-api-access-sfkjr\") pod \"test-curl-7264-140675705897648\" (UID: \"d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8\") " pod="openshift-ingress/test-curl-7264-140675705897648"
Jul 14 12:04:28.126098 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:28.125996    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705897648"
Jul 14 12:04:28.259735 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:28.259706    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705897648"]
Jul 14 12:04:28.261496 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:04:28.261460    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd07d0dfb_2a0a_40a3_9c3c_3b3edb4166f8.slice/crio-ed756835c9e7e66d8764637f3db2149b0b046a60937ea7cb4ef0918ceff5bd8f WatchSource:0}: Error finding container ed756835c9e7e66d8764637f3db2149b0b046a60937ea7cb4ef0918ceff5bd8f: Status 404 returned error can't find the container with id ed756835c9e7e66d8764637f3db2149b0b046a60937ea7cb4ef0918ceff5bd8f
Jul 14 12:04:28.262875 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:28.262858    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 12:04:28.472067 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:28.471981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705897648" event={"ID":"d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8","Type":"ContainerStarted","Data":"ed756835c9e7e66d8764637f3db2149b0b046a60937ea7cb4ef0918ceff5bd8f"}
Jul 14 12:04:29.477996 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:29.477960    2564 generic.go:356] "Generic (PLEG): container finished" podID="d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8" containerID="4a4be7c69e901a94f034f97b49ba0e174355599844e8caca5405129eb4182b1c" exitCode=0
Jul 14 12:04:29.478379 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:29.478048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705897648" event={"ID":"d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8","Type":"ContainerDied","Data":"4a4be7c69e901a94f034f97b49ba0e174355599844e8caca5405129eb4182b1c"}
Jul 14 12:04:30.619999 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.619974    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705897648"
Jul 14 12:04:30.654734 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.654677    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7264-140675705897648_d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8/test-curl-7264-140675705897648/0.log"
Jul 14 12:04:30.674911 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.674878    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7264-140675705897648"]
Jul 14 12:04:30.680732 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.680707    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7264-140675705897648"]
Jul 14 12:04:30.742515 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.742445    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfkjr\" (UniqueName: \"kubernetes.io/projected/d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8-kube-api-access-sfkjr\") pod \"d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8\" (UID: \"d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8\") "
Jul 14 12:04:30.744713 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.744675    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8-kube-api-access-sfkjr" (OuterVolumeSpecName: "kube-api-access-sfkjr") pod "d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8" (UID: "d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8"). InnerVolumeSpecName "kube-api-access-sfkjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:04:30.769801 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.769778    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8" path="/var/lib/kubelet/pods/d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8/volumes"
Jul 14 12:04:30.770130 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.770116    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:04:30.841504 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.841470    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:30.841825 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.841812    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8" containerName="test-curl-7264-140675705897648"
Jul 14 12:04:30.841869 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.841826    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8" containerName="test-curl-7264-140675705897648"
Jul 14 12:04:30.841906 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.841893    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8" containerName="test-curl-7264-140675705897648"
Jul 14 12:04:30.843617 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.843596    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfkjr\" (UniqueName: \"kubernetes.io/projected/d07d0dfb-2a0a-40a3-9c3c-3b3edb4166f8-kube-api-access-sfkjr\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:30.844975 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.844959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:30.851296 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.851270    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:30.944860 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:30.944823    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5mqm\" (UniqueName: \"kubernetes.io/projected/b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9-kube-api-access-r5mqm\") pod \"test-curl-7264-140675705941280\" (UID: \"b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9\") " pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:31.046212 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:31.046184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5mqm\" (UniqueName: \"kubernetes.io/projected/b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9-kube-api-access-r5mqm\") pod \"test-curl-7264-140675705941280\" (UID: \"b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9\") " pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:31.054782 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:31.054751    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5mqm\" (UniqueName: \"kubernetes.io/projected/b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9-kube-api-access-r5mqm\") pod \"test-curl-7264-140675705941280\" (UID: \"b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9\") " pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:31.154652 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:31.154617    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:31.283026 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:31.282993    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:31.284722 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:04:31.284674    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4a4c80c_1e2b_408f_b5e9_83ee3ed67be9.slice/crio-774bd899ae4c7e8a59b5b357f280de620e5a193bbda8d0921b59ab5a012aea12 WatchSource:0}: Error finding container 774bd899ae4c7e8a59b5b357f280de620e5a193bbda8d0921b59ab5a012aea12: Status 404 returned error can't find the container with id 774bd899ae4c7e8a59b5b357f280de620e5a193bbda8d0921b59ab5a012aea12
Jul 14 12:04:31.488155 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:31.488113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705941280" event={"ID":"b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9","Type":"ContainerStarted","Data":"774bd899ae4c7e8a59b5b357f280de620e5a193bbda8d0921b59ab5a012aea12"}
Jul 14 12:04:31.489232 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:31.489213    2564 scope.go:117] "RemoveContainer" containerID="4a4be7c69e901a94f034f97b49ba0e174355599844e8caca5405129eb4182b1c"
Jul 14 12:04:31.489232 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:31.489226    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705897648"
Jul 14 12:04:32.495137 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:32.495101    2564 generic.go:356] "Generic (PLEG): container finished" podID="b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9" containerID="e714cd5accebda496a28b6af26ceb68f60664d4678f4b4e98e59602058d3ee61" exitCode=0
Jul 14 12:04:32.495676 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:32.495189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705941280" event={"ID":"b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9","Type":"ContainerDied","Data":"e714cd5accebda496a28b6af26ceb68f60664d4678f4b4e98e59602058d3ee61"}
Jul 14 12:04:33.644035 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.644008    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:33.667240 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.667218    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7264-140675705941280_b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9/test-curl-7264-140675705941280/0.log"
Jul 14 12:04:33.693038 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.693007    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:33.699926 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.699902    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:33.770301 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.770270    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5mqm\" (UniqueName: \"kubernetes.io/projected/b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9-kube-api-access-r5mqm\") pod \"b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9\" (UID: \"b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9\") "
Jul 14 12:04:33.772334 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.772298    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9-kube-api-access-r5mqm" (OuterVolumeSpecName: "kube-api-access-r5mqm") pod "b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9" (UID: "b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9"). InnerVolumeSpecName "kube-api-access-r5mqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:04:33.827152 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.827114    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:33.827472 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.827459    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9" containerName="test-curl-7264-140675705941280"
Jul 14 12:04:33.827519 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.827474    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9" containerName="test-curl-7264-140675705941280"
Jul 14 12:04:33.827557 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.827537    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9" containerName="test-curl-7264-140675705941280"
Jul 14 12:04:33.830793 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.830773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:33.835070 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.835043    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:33.871465 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.871437    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5mqm\" (UniqueName: \"kubernetes.io/projected/b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9-kube-api-access-r5mqm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:33.972173 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:33.972129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2k2k\" (UniqueName: \"kubernetes.io/projected/18dc7d4f-3cac-492b-8aac-6226174da05a-kube-api-access-v2k2k\") pod \"test-curl-7264-140675705941280\" (UID: \"18dc7d4f-3cac-492b-8aac-6226174da05a\") " pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:34.073651 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.073557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2k2k\" (UniqueName: \"kubernetes.io/projected/18dc7d4f-3cac-492b-8aac-6226174da05a-kube-api-access-v2k2k\") pod \"test-curl-7264-140675705941280\" (UID: \"18dc7d4f-3cac-492b-8aac-6226174da05a\") " pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:34.082064 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.082037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2k2k\" (UniqueName: \"kubernetes.io/projected/18dc7d4f-3cac-492b-8aac-6226174da05a-kube-api-access-v2k2k\") pod \"test-curl-7264-140675705941280\" (UID: \"18dc7d4f-3cac-492b-8aac-6226174da05a\") " pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:34.141146 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.141110    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:34.271011 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.270987    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:34.273513 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:04:34.273490    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18dc7d4f_3cac_492b_8aac_6226174da05a.slice/crio-8eb87b13e265556d8e1fbfeb944672e487c5902c043eb3987c100f2c148bc560 WatchSource:0}: Error finding container 8eb87b13e265556d8e1fbfeb944672e487c5902c043eb3987c100f2c148bc560: Status 404 returned error can't find the container with id 8eb87b13e265556d8e1fbfeb944672e487c5902c043eb3987c100f2c148bc560
Jul 14 12:04:34.508272 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.508186    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:34.508272 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.508185    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="774bd899ae4c7e8a59b5b357f280de620e5a193bbda8d0921b59ab5a012aea12"
Jul 14 12:04:34.509364 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.509336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705941280" event={"ID":"18dc7d4f-3cac-492b-8aac-6226174da05a","Type":"ContainerStarted","Data":"8eb87b13e265556d8e1fbfeb944672e487c5902c043eb3987c100f2c148bc560"}
Jul 14 12:04:34.519541 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.519510    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7264-140675705941280" oldPodUID="b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9" podUID="18dc7d4f-3cac-492b-8aac-6226174da05a"
Jul 14 12:04:34.769125 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:34.769096    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9" path="/var/lib/kubelet/pods/b4a4c80c-1e2b-408f-b5e9-83ee3ed67be9/volumes"
Jul 14 12:04:35.515636 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:35.515603    2564 generic.go:356] "Generic (PLEG): container finished" podID="18dc7d4f-3cac-492b-8aac-6226174da05a" containerID="0b7c51f6b0070826120e14c58b7106a4513d48a361e97a1d04473c977568f3c2" exitCode=0
Jul 14 12:04:35.515846 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:35.515710    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675705941280" event={"ID":"18dc7d4f-3cac-492b-8aac-6226174da05a","Type":"ContainerDied","Data":"0b7c51f6b0070826120e14c58b7106a4513d48a361e97a1d04473c977568f3c2"}
Jul 14 12:04:36.669301 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.669274    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:36.688126 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.688094    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7264-140675705941280_18dc7d4f-3cac-492b-8aac-6226174da05a/test-curl-7264-140675705941280/0.log"
Jul 14 12:04:36.712019 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.711988    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:36.719673 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.719638    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7264-140675705941280"]
Jul 14 12:04:36.796746 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.796624    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2k2k\" (UniqueName: \"kubernetes.io/projected/18dc7d4f-3cac-492b-8aac-6226174da05a-kube-api-access-v2k2k\") pod \"18dc7d4f-3cac-492b-8aac-6226174da05a\" (UID: \"18dc7d4f-3cac-492b-8aac-6226174da05a\") "
Jul 14 12:04:36.798828 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.798790    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18dc7d4f-3cac-492b-8aac-6226174da05a-kube-api-access-v2k2k" (OuterVolumeSpecName: "kube-api-access-v2k2k") pod "18dc7d4f-3cac-492b-8aac-6226174da05a" (UID: "18dc7d4f-3cac-492b-8aac-6226174da05a"). InnerVolumeSpecName "kube-api-access-v2k2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:04:36.846202 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.846159    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:36.846526 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.846514    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18dc7d4f-3cac-492b-8aac-6226174da05a" containerName="test-curl-7264-140675705941280"
Jul 14 12:04:36.846572 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.846528    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="18dc7d4f-3cac-492b-8aac-6226174da05a" containerName="test-curl-7264-140675705941280"
Jul 14 12:04:36.846605 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.846587    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="18dc7d4f-3cac-492b-8aac-6226174da05a" containerName="test-curl-7264-140675705941280"
Jul 14 12:04:36.849209 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.849191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:36.855329 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.855299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:36.897685 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.897639    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2k2k\" (UniqueName: \"kubernetes.io/projected/18dc7d4f-3cac-492b-8aac-6226174da05a-kube-api-access-v2k2k\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:36.998547 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:36.998499    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d5gr\" (UniqueName: \"kubernetes.io/projected/f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a-kube-api-access-6d5gr\") pod \"test-curl-7264-140675706704208\" (UID: \"f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a\") " pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:37.099765 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:37.099657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d5gr\" (UniqueName: \"kubernetes.io/projected/f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a-kube-api-access-6d5gr\") pod \"test-curl-7264-140675706704208\" (UID: \"f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a\") " pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:37.107685 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:37.107661    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d5gr\" (UniqueName: \"kubernetes.io/projected/f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a-kube-api-access-6d5gr\") pod \"test-curl-7264-140675706704208\" (UID: \"f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a\") " pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:37.159235 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:37.159201    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:37.290562 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:37.290536    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:37.294593 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:04:37.294564    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8d7b2f5_e58b_4a82_bdb4_eb1ba6ba870a.slice/crio-a6f3f9b42949c3ec606203d9f33e0bb54458df4c2951698ebe4e57369f8fe111 WatchSource:0}: Error finding container a6f3f9b42949c3ec606203d9f33e0bb54458df4c2951698ebe4e57369f8fe111: Status 404 returned error can't find the container with id a6f3f9b42949c3ec606203d9f33e0bb54458df4c2951698ebe4e57369f8fe111
Jul 14 12:04:37.528543 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:37.528517    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675705941280"
Jul 14 12:04:37.528751 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:37.528526    2564 scope.go:117] "RemoveContainer" containerID="0b7c51f6b0070826120e14c58b7106a4513d48a361e97a1d04473c977568f3c2"
Jul 14 12:04:37.529750 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:37.529690    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675706704208" event={"ID":"f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a","Type":"ContainerStarted","Data":"a6f3f9b42949c3ec606203d9f33e0bb54458df4c2951698ebe4e57369f8fe111"}
Jul 14 12:04:38.129878 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.129836    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn" podUID="d9fa83bd-4b26-4c8d-8602-e966b1324e95" containerName="main" containerID="cri-o://5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7" gracePeriod=45
Jul 14 12:04:38.371120 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.371096    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:04:38.414134 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414052    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tmp-dir\") pod \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") "
Jul 14 12:04:38.414134 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414121    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctv2h\" (UniqueName: \"kubernetes.io/projected/d9fa83bd-4b26-4c8d-8602-e966b1324e95-kube-api-access-ctv2h\") pod \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") "
Jul 14 12:04:38.414326 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414184    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-model-cache\") pod \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") "
Jul 14 12:04:38.414326 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414256    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-dshm\") pod \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") "
Jul 14 12:04:38.414326 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414299    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d9fa83bd-4b26-4c8d-8602-e966b1324e95" (UID: "d9fa83bd-4b26-4c8d-8602-e966b1324e95"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:38.414488 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414456    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-model-cache" (OuterVolumeSpecName: "model-cache") pod "d9fa83bd-4b26-4c8d-8602-e966b1324e95" (UID: "d9fa83bd-4b26-4c8d-8602-e966b1324e95"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:38.414581 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414561    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:38.414637 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.414583    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:38.416248 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.416222    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9fa83bd-4b26-4c8d-8602-e966b1324e95-kube-api-access-ctv2h" (OuterVolumeSpecName: "kube-api-access-ctv2h") pod "d9fa83bd-4b26-4c8d-8602-e966b1324e95" (UID: "d9fa83bd-4b26-4c8d-8602-e966b1324e95"). InnerVolumeSpecName "kube-api-access-ctv2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:04:38.416248 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.416233    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-dshm" (OuterVolumeSpecName: "dshm") pod "d9fa83bd-4b26-4c8d-8602-e966b1324e95" (UID: "d9fa83bd-4b26-4c8d-8602-e966b1324e95"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:38.514999 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.514968    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-home\") pod \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") "
Jul 14 12:04:38.515197 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.515010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tls-certs\") pod \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\" (UID: \"d9fa83bd-4b26-4c8d-8602-e966b1324e95\") "
Jul 14 12:04:38.515261 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.515226    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-home" (OuterVolumeSpecName: "home") pod "d9fa83bd-4b26-4c8d-8602-e966b1324e95" (UID: "d9fa83bd-4b26-4c8d-8602-e966b1324e95"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:38.515341 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.515327    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctv2h\" (UniqueName: \"kubernetes.io/projected/d9fa83bd-4b26-4c8d-8602-e966b1324e95-kube-api-access-ctv2h\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:38.515381 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.515344    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:38.515381 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.515353    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9fa83bd-4b26-4c8d-8602-e966b1324e95-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:38.517035 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.517015    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d9fa83bd-4b26-4c8d-8602-e966b1324e95" (UID: "d9fa83bd-4b26-4c8d-8602-e966b1324e95"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 12:04:38.542314 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.542285    2564 generic.go:356] "Generic (PLEG): container finished" podID="d9fa83bd-4b26-4c8d-8602-e966b1324e95" containerID="5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7" exitCode=0
Jul 14 12:04:38.542497 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.542346    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"
Jul 14 12:04:38.542497 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.542368    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn" event={"ID":"d9fa83bd-4b26-4c8d-8602-e966b1324e95","Type":"ContainerDied","Data":"5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7"}
Jul 14 12:04:38.542497 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.542413    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn" event={"ID":"d9fa83bd-4b26-4c8d-8602-e966b1324e95","Type":"ContainerDied","Data":"41fc62e042d65fbb902ddf272f90dafcf9ac05e4c7ee25c56b47eb52f8395560"}
Jul 14 12:04:38.542497 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.542436    2564 scope.go:117] "RemoveContainer" containerID="5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7"
Jul 14 12:04:38.544956 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.544923    2564 generic.go:356] "Generic (PLEG): container finished" podID="f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a" containerID="572b288337d861420a3f9515e691eaba14bb8e617d11cab4b4d2b939413ec6db" exitCode=0
Jul 14 12:04:38.545060 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.545013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675706704208" event={"ID":"f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a","Type":"ContainerDied","Data":"572b288337d861420a3f9515e691eaba14bb8e617d11cab4b4d2b939413ec6db"}
Jul 14 12:04:38.551489 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.551467    2564 scope.go:117] "RemoveContainer" containerID="5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7"
Jul 14 12:04:38.551756 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:04:38.551736    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7\": container with ID starting with 5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7 not found: ID does not exist" containerID="5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7"
Jul 14 12:04:38.551826 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.551763    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7"} err="failed to get container status \"5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7\": rpc error: code = NotFound desc = could not find container \"5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7\": container with ID starting with 5f32876d0732188aec1d7db56119c753b8ff859190665b8cb595872166d4fda7 not found: ID does not exist"
Jul 14 12:04:38.570536 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.570504    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"]
Jul 14 12:04:38.575013 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.574992    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1cc796/rate-test-model-1cc796-kserve-d994d856c-bzbcn"]
Jul 14 12:04:38.616579 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.616545    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9fa83bd-4b26-4c8d-8602-e966b1324e95-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:38.769999 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.769970    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18dc7d4f-3cac-492b-8aac-6226174da05a" path="/var/lib/kubelet/pods/18dc7d4f-3cac-492b-8aac-6226174da05a/volumes"
Jul 14 12:04:38.770348 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:38.770335    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9fa83bd-4b26-4c8d-8602-e966b1324e95" path="/var/lib/kubelet/pods/d9fa83bd-4b26-4c8d-8602-e966b1324e95/volumes"
Jul 14 12:04:39.096840 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.096747    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq" podUID="be5e175d-b84e-4701-9772-2c3659cdf10d" containerName="main" containerID="cri-o://e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930" gracePeriod=45
Jul 14 12:04:39.356960 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.356896    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:04:39.422716 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.422660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be5e175d-b84e-4701-9772-2c3659cdf10d-tls-certs\") pod \"be5e175d-b84e-4701-9772-2c3659cdf10d\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") "
Jul 14 12:04:39.422901 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.422750    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-home\") pod \"be5e175d-b84e-4701-9772-2c3659cdf10d\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") "
Jul 14 12:04:39.422901 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.422770    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh9b7\" (UniqueName: \"kubernetes.io/projected/be5e175d-b84e-4701-9772-2c3659cdf10d-kube-api-access-hh9b7\") pod \"be5e175d-b84e-4701-9772-2c3659cdf10d\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") "
Jul 14 12:04:39.422901 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.422789    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-dshm\") pod \"be5e175d-b84e-4701-9772-2c3659cdf10d\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") "
Jul 14 12:04:39.422901 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.422816    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-tmp-dir\") pod \"be5e175d-b84e-4701-9772-2c3659cdf10d\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") "
Jul 14 12:04:39.422901 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.422834    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-model-cache\") pod \"be5e175d-b84e-4701-9772-2c3659cdf10d\" (UID: \"be5e175d-b84e-4701-9772-2c3659cdf10d\") "
Jul 14 12:04:39.423161 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.423103    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-home" (OuterVolumeSpecName: "home") pod "be5e175d-b84e-4701-9772-2c3659cdf10d" (UID: "be5e175d-b84e-4701-9772-2c3659cdf10d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:39.423161 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.423130    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "be5e175d-b84e-4701-9772-2c3659cdf10d" (UID: "be5e175d-b84e-4701-9772-2c3659cdf10d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:39.423286 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.423260    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-model-cache" (OuterVolumeSpecName: "model-cache") pod "be5e175d-b84e-4701-9772-2c3659cdf10d" (UID: "be5e175d-b84e-4701-9772-2c3659cdf10d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:39.424850 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.424828    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be5e175d-b84e-4701-9772-2c3659cdf10d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "be5e175d-b84e-4701-9772-2c3659cdf10d" (UID: "be5e175d-b84e-4701-9772-2c3659cdf10d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 12:04:39.424941 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.424888    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-dshm" (OuterVolumeSpecName: "dshm") pod "be5e175d-b84e-4701-9772-2c3659cdf10d" (UID: "be5e175d-b84e-4701-9772-2c3659cdf10d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 12:04:39.425045 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.425028    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5e175d-b84e-4701-9772-2c3659cdf10d-kube-api-access-hh9b7" (OuterVolumeSpecName: "kube-api-access-hh9b7") pod "be5e175d-b84e-4701-9772-2c3659cdf10d" (UID: "be5e175d-b84e-4701-9772-2c3659cdf10d"). InnerVolumeSpecName "kube-api-access-hh9b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:04:39.523766 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.523716    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be5e175d-b84e-4701-9772-2c3659cdf10d-tls-certs\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:39.523766 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.523760    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-home\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:39.523766 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.523774    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh9b7\" (UniqueName: \"kubernetes.io/projected/be5e175d-b84e-4701-9772-2c3659cdf10d-kube-api-access-hh9b7\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:39.524012 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.523789    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-dshm\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:39.524012 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.523803    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-tmp-dir\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:39.524012 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.523814    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be5e175d-b84e-4701-9772-2c3659cdf10d-model-cache\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:39.552768 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.552737    2564 generic.go:356] "Generic (PLEG): container finished" podID="be5e175d-b84e-4701-9772-2c3659cdf10d" containerID="e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930" exitCode=0
Jul 14 12:04:39.552926 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.552782    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"
Jul 14 12:04:39.552926 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.552831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq" event={"ID":"be5e175d-b84e-4701-9772-2c3659cdf10d","Type":"ContainerDied","Data":"e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930"}
Jul 14 12:04:39.552926 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.552877    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq" event={"ID":"be5e175d-b84e-4701-9772-2c3659cdf10d","Type":"ContainerDied","Data":"ef74eb42a880eac70630c3eb41fe61542dbf0d42594c89f6b1f435b5fe46faf6"}
Jul 14 12:04:39.552926 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.552898    2564 scope.go:117] "RemoveContainer" containerID="e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930"
Jul 14 12:04:39.562167 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.562147    2564 scope.go:117] "RemoveContainer" containerID="e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930"
Jul 14 12:04:39.562441 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:04:39.562424    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930\": container with ID starting with e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930 not found: ID does not exist" containerID="e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930"
Jul 14 12:04:39.562514 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.562448    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930"} err="failed to get container status \"e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930\": rpc error: code = NotFound desc = could not find container \"e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930\": container with ID starting with e2a8e2052f957a867b7e2b42c766e879f3cb7bc139227384182b9a0707602930 not found: ID does not exist"
Jul 14 12:04:39.574874 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.574844    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"]
Jul 14 12:04:39.579984 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.579957    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0508a3/rate-test-model-0508a3-kserve-56c5bd8687-k9dbq"]
Jul 14 12:04:39.687210 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.687186    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:39.709677 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.709648    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7264-140675706704208_f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a/test-curl-7264-140675706704208/0.log"
Jul 14 12:04:39.724933 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.724907    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d5gr\" (UniqueName: \"kubernetes.io/projected/f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a-kube-api-access-6d5gr\") pod \"f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a\" (UID: \"f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a\") "
Jul 14 12:04:39.726962 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.726931    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a-kube-api-access-6d5gr" (OuterVolumeSpecName: "kube-api-access-6d5gr") pod "f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a" (UID: "f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a"). InnerVolumeSpecName "kube-api-access-6d5gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:04:39.734161 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.734133    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:39.741495 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.741471    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:39.826439 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.826392    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d5gr\" (UniqueName: \"kubernetes.io/projected/f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a-kube-api-access-6d5gr\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:39.874234 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874156    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:39.874491 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874479    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a" containerName="test-curl-7264-140675706704208"
Jul 14 12:04:39.874530 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874492    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a" containerName="test-curl-7264-140675706704208"
Jul 14 12:04:39.874530 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874507    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be5e175d-b84e-4701-9772-2c3659cdf10d" containerName="main"
Jul 14 12:04:39.874530 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874512    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="be5e175d-b84e-4701-9772-2c3659cdf10d" containerName="main"
Jul 14 12:04:39.874530 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874524    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9fa83bd-4b26-4c8d-8602-e966b1324e95" containerName="main"
Jul 14 12:04:39.874714 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874531    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9fa83bd-4b26-4c8d-8602-e966b1324e95" containerName="main"
Jul 14 12:04:39.874714 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874584    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="be5e175d-b84e-4701-9772-2c3659cdf10d" containerName="main"
Jul 14 12:04:39.874714 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874593    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9fa83bd-4b26-4c8d-8602-e966b1324e95" containerName="main"
Jul 14 12:04:39.874714 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.874600    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a" containerName="test-curl-7264-140675706704208"
Jul 14 12:04:39.877425 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.877407    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:39.883544 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.883519    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:39.926771 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:39.926739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqncn\" (UniqueName: \"kubernetes.io/projected/e92f9e09-8625-4a39-b689-4f4595797011-kube-api-access-mqncn\") pod \"test-curl-7264-140675706704208\" (UID: \"e92f9e09-8625-4a39-b689-4f4595797011\") " pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:40.027952 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.027902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqncn\" (UniqueName: \"kubernetes.io/projected/e92f9e09-8625-4a39-b689-4f4595797011-kube-api-access-mqncn\") pod \"test-curl-7264-140675706704208\" (UID: \"e92f9e09-8625-4a39-b689-4f4595797011\") " pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:40.036687 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.036654    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqncn\" (UniqueName: \"kubernetes.io/projected/e92f9e09-8625-4a39-b689-4f4595797011-kube-api-access-mqncn\") pod \"test-curl-7264-140675706704208\" (UID: \"e92f9e09-8625-4a39-b689-4f4595797011\") " pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:40.187418 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.187328    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:40.320137 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.320106    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:40.321846 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:04:40.321815    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode92f9e09_8625_4a39_b689_4f4595797011.slice/crio-bc114f310cadd0693abbf4941a8480566d2ea60f47d3841ef7d874f3505ada49 WatchSource:0}: Error finding container bc114f310cadd0693abbf4941a8480566d2ea60f47d3841ef7d874f3505ada49: Status 404 returned error can't find the container with id bc114f310cadd0693abbf4941a8480566d2ea60f47d3841ef7d874f3505ada49
Jul 14 12:04:40.559351 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.559314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675706704208" event={"ID":"e92f9e09-8625-4a39-b689-4f4595797011","Type":"ContainerStarted","Data":"bc114f310cadd0693abbf4941a8480566d2ea60f47d3841ef7d874f3505ada49"}
Jul 14 12:04:40.561310 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.561288    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:40.561434 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.561295    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6f3f9b42949c3ec606203d9f33e0bb54458df4c2951698ebe4e57369f8fe111"
Jul 14 12:04:40.572643 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.572614    2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7264-140675706704208" oldPodUID="f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a" podUID="e92f9e09-8625-4a39-b689-4f4595797011"
Jul 14 12:04:40.770604 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.770573    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be5e175d-b84e-4701-9772-2c3659cdf10d" path="/var/lib/kubelet/pods/be5e175d-b84e-4701-9772-2c3659cdf10d/volumes"
Jul 14 12:04:40.770974 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:40.770959    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a" path="/var/lib/kubelet/pods/f8d7b2f5-e58b-4a82-bdb4-eb1ba6ba870a/volumes"
Jul 14 12:04:41.567185 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:41.567150    2564 generic.go:356] "Generic (PLEG): container finished" podID="e92f9e09-8625-4a39-b689-4f4595797011" containerID="6edece8fd2be5c95cb66d68ccae2d82bea9c1a6a9d30099330d79b4b5f4c5fae" exitCode=0
Jul 14 12:04:41.567541 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:41.567240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7264-140675706704208" event={"ID":"e92f9e09-8625-4a39-b689-4f4595797011","Type":"ContainerDied","Data":"6edece8fd2be5c95cb66d68ccae2d82bea9c1a6a9d30099330d79b4b5f4c5fae"}
Jul 14 12:04:42.715881 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.715849    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:42.736022 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.735996    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7264-140675706704208_e92f9e09-8625-4a39-b689-4f4595797011/test-curl-7264-140675706704208/0.log"
Jul 14 12:04:42.750248 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.750224    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqncn\" (UniqueName: \"kubernetes.io/projected/e92f9e09-8625-4a39-b689-4f4595797011-kube-api-access-mqncn\") pod \"e92f9e09-8625-4a39-b689-4f4595797011\" (UID: \"e92f9e09-8625-4a39-b689-4f4595797011\") "
Jul 14 12:04:42.752244 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.752219    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92f9e09-8625-4a39-b689-4f4595797011-kube-api-access-mqncn" (OuterVolumeSpecName: "kube-api-access-mqncn") pod "e92f9e09-8625-4a39-b689-4f4595797011" (UID: "e92f9e09-8625-4a39-b689-4f4595797011"). InnerVolumeSpecName "kube-api-access-mqncn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 12:04:42.760104 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.760081    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:42.769227 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.769208    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7264-140675706704208"]
Jul 14 12:04:42.851013 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.850928    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqncn\" (UniqueName: \"kubernetes.io/projected/e92f9e09-8625-4a39-b689-4f4595797011-kube-api-access-mqncn\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:42.960890 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.960850    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"]
Jul 14 12:04:42.961255 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.961237    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e92f9e09-8625-4a39-b689-4f4595797011" containerName="test-curl-7264-140675706704208"
Jul 14 12:04:42.961335 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.961258    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92f9e09-8625-4a39-b689-4f4595797011" containerName="test-curl-7264-140675706704208"
Jul 14 12:04:42.961395 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.961378    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e92f9e09-8625-4a39-b689-4f4595797011" containerName="test-curl-7264-140675706704208"
Jul 14 12:04:42.964399 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.964377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"
Jul 14 12:04:42.966578 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.966558    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 12:04:42.966677 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.966614    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 12:04:42.971234 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:42.971212    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"]
Jul 14 12:04:43.052502 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.052450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41c54d39-f9fb-40f5-9056-afd63d7c6dc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4\" (UID: \"41c54d39-f9fb-40f5-9056-afd63d7c6dc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"
Jul 14 12:04:43.153545 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.153436    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41c54d39-f9fb-40f5-9056-afd63d7c6dc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4\" (UID: \"41c54d39-f9fb-40f5-9056-afd63d7c6dc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"
Jul 14 12:04:43.154150 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.154125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41c54d39-f9fb-40f5-9056-afd63d7c6dc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4\" (UID: \"41c54d39-f9fb-40f5-9056-afd63d7c6dc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"
Jul 14 12:04:43.275908 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.275874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"
Jul 14 12:04:43.403140 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.403113    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"]
Jul 14 12:04:43.404617 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:04:43.404559    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c54d39_f9fb_40f5_9056_afd63d7c6dc3.slice/crio-50ed1d4b828c1dc16af74b40f257675d2ade5e7d931ddad5c936f19e7216ead9 WatchSource:0}: Error finding container 50ed1d4b828c1dc16af74b40f257675d2ade5e7d931ddad5c936f19e7216ead9: Status 404 returned error can't find the container with id 50ed1d4b828c1dc16af74b40f257675d2ade5e7d931ddad5c936f19e7216ead9
Jul 14 12:04:43.578630 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.578596    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:43.578791 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.578604    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc114f310cadd0693abbf4941a8480566d2ea60f47d3841ef7d874f3505ada49"
Jul 14 12:04:43.579800 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:43.579777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4" event={"ID":"41c54d39-f9fb-40f5-9056-afd63d7c6dc3","Type":"ContainerStarted","Data":"50ed1d4b828c1dc16af74b40f257675d2ade5e7d931ddad5c936f19e7216ead9"}
Jul 14 12:04:43.585512 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:04:43.585474    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7264-140675706704208\" is forbidden: User \"system:node:ip-10-0-142-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-172.ec2.internal' and this object" podUID="e92f9e09-8625-4a39-b689-4f4595797011" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:44.586475 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:44.586438    2564 generic.go:356] "Generic (PLEG): container finished" podID="41c54d39-f9fb-40f5-9056-afd63d7c6dc3" containerID="d9024c1f92fc87c39363a4b2879e0304ece06d64d925fdbd1a1a852b976a4ed8" exitCode=0
Jul 14 12:04:44.586946 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:44.586476    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4" event={"ID":"41c54d39-f9fb-40f5-9056-afd63d7c6dc3","Type":"ContainerDied","Data":"d9024c1f92fc87c39363a4b2879e0304ece06d64d925fdbd1a1a852b976a4ed8"}
Jul 14 12:04:44.588623 ip-10-0-142-172 kubenswrapper[2564]: E0714 12:04:44.588598    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7264-140675706704208\" is forbidden: User \"system:node:ip-10-0-142-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-172.ec2.internal' and this object" podUID="e92f9e09-8625-4a39-b689-4f4595797011" pod="openshift-ingress/test-curl-7264-140675706704208"
Jul 14 12:04:44.769132 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:44.769096    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e92f9e09-8625-4a39-b689-4f4595797011" path="/var/lib/kubelet/pods/e92f9e09-8625-4a39-b689-4f4595797011/volumes"
Jul 14 12:04:45.720017 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:45.719993    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"
Jul 14 12:04:45.774346 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:45.774314    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41c54d39-f9fb-40f5-9056-afd63d7c6dc3-maas-api-service-ca\") pod \"41c54d39-f9fb-40f5-9056-afd63d7c6dc3\" (UID: \"41c54d39-f9fb-40f5-9056-afd63d7c6dc3\") "
Jul 14 12:04:45.774664 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:45.774644    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41c54d39-f9fb-40f5-9056-afd63d7c6dc3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41c54d39-f9fb-40f5-9056-afd63d7c6dc3" (UID: "41c54d39-f9fb-40f5-9056-afd63d7c6dc3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 12:04:45.875291 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:45.875193    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41c54d39-f9fb-40f5-9056-afd63d7c6dc3-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:04:46.597712 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:46.597669    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4"
Jul 14 12:04:46.597884 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:46.597726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4" event={"ID":"41c54d39-f9fb-40f5-9056-afd63d7c6dc3","Type":"ContainerDied","Data":"50ed1d4b828c1dc16af74b40f257675d2ade5e7d931ddad5c936f19e7216ead9"}
Jul 14 12:04:46.597884 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:46.597773    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50ed1d4b828c1dc16af74b40f257675d2ade5e7d931ddad5c936f19e7216ead9"
Jul 14 12:04:53.645578 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:53.645495    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"]
Jul 14 12:04:53.649048 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:53.649020    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-1cc796-29733840-59lhv"]
Jul 14 12:04:54.770046 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:54.770011    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb65ac07-074f-4ed1-a0c4-325550d72ea2" path="/var/lib/kubelet/pods/eb65ac07-074f-4ed1-a0c4-325550d72ea2/volumes"
Jul 14 12:04:57.263876 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:04:57.263844    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:05:09.165414 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.165378    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"]
Jul 14 12:05:09.167767 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.165768    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41c54d39-f9fb-40f5-9056-afd63d7c6dc3" containerName="revoke-keys"
Jul 14 12:05:09.167767 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.165783    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c54d39-f9fb-40f5-9056-afd63d7c6dc3" containerName="revoke-keys"
Jul 14 12:05:09.167767 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.165853    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="41c54d39-f9fb-40f5-9056-afd63d7c6dc3" containerName="revoke-keys"
Jul 14 12:05:09.168570 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.168554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"
Jul 14 12:05:09.170948 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.170924    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 12:05:09.171073 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.170984    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lxbn\""
Jul 14 12:05:09.176612 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.176585    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"]
Jul 14 12:05:09.274250 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.274212    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b703eed-bb82-4ffc-9ddf-9e51bf31458c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v\" (UID: \"9b703eed-bb82-4ffc-9ddf-9e51bf31458c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"
Jul 14 12:05:09.375518 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.375475    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b703eed-bb82-4ffc-9ddf-9e51bf31458c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v\" (UID: \"9b703eed-bb82-4ffc-9ddf-9e51bf31458c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"
Jul 14 12:05:09.376183 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.376156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b703eed-bb82-4ffc-9ddf-9e51bf31458c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v\" (UID: \"9b703eed-bb82-4ffc-9ddf-9e51bf31458c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"
Jul 14 12:05:09.480393 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.480290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"
Jul 14 12:05:09.616024 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.615991    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"]
Jul 14 12:05:09.617654 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:05:09.617626    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b703eed_bb82_4ffc_9ddf_9e51bf31458c.slice/crio-5fb9e7efd452be3299d8cee0d8b4b8d2da97b45be50eb0c24a929a44252b761b WatchSource:0}: Error finding container 5fb9e7efd452be3299d8cee0d8b4b8d2da97b45be50eb0c24a929a44252b761b: Status 404 returned error can't find the container with id 5fb9e7efd452be3299d8cee0d8b4b8d2da97b45be50eb0c24a929a44252b761b
Jul 14 12:05:09.724677 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:09.724643    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v" event={"ID":"9b703eed-bb82-4ffc-9ddf-9e51bf31458c","Type":"ContainerStarted","Data":"5fb9e7efd452be3299d8cee0d8b4b8d2da97b45be50eb0c24a929a44252b761b"}
Jul 14 12:05:10.731197 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:10.731108    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b703eed-bb82-4ffc-9ddf-9e51bf31458c" containerID="6341c26366a4d889cde0b281f1ed6c273a2d0f5eaf20d88e59f46d2ae19fa8a2" exitCode=0
Jul 14 12:05:10.731558 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:10.731202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v" event={"ID":"9b703eed-bb82-4ffc-9ddf-9e51bf31458c","Type":"ContainerDied","Data":"6341c26366a4d889cde0b281f1ed6c273a2d0f5eaf20d88e59f46d2ae19fa8a2"}
Jul 14 12:05:11.873388 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:11.873360    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"
Jul 14 12:05:11.999222 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:11.999137    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b703eed-bb82-4ffc-9ddf-9e51bf31458c-maas-api-service-ca\") pod \"9b703eed-bb82-4ffc-9ddf-9e51bf31458c\" (UID: \"9b703eed-bb82-4ffc-9ddf-9e51bf31458c\") "
Jul 14 12:05:11.999487 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:11.999462    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b703eed-bb82-4ffc-9ddf-9e51bf31458c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b703eed-bb82-4ffc-9ddf-9e51bf31458c" (UID: "9b703eed-bb82-4ffc-9ddf-9e51bf31458c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 12:05:12.100407 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:12.100372    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b703eed-bb82-4ffc-9ddf-9e51bf31458c-maas-api-service-ca\") on node \"ip-10-0-142-172.ec2.internal\" DevicePath \"\""
Jul 14 12:05:12.750838 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:12.750807    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v"
Jul 14 12:05:12.751045 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:12.750809    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v" event={"ID":"9b703eed-bb82-4ffc-9ddf-9e51bf31458c","Type":"ContainerDied","Data":"5fb9e7efd452be3299d8cee0d8b4b8d2da97b45be50eb0c24a929a44252b761b"}
Jul 14 12:05:12.751045 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:12.750973    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fb9e7efd452be3299d8cee0d8b4b8d2da97b45be50eb0c24a929a44252b761b"
Jul 14 12:05:23.275797 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:23.275751    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b6pjt"]
Jul 14 12:05:35.632084 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:35.632028    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-97f67d9-g9wmz_52f31154-5875-44bc-86e9-514da0d7db6f/authorino/0.log"
Jul 14 12:05:41.091042 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:41.091006    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-ld599_e223985c-991b-4f3e-836f-8fbd992e4dea/copy-manifests/0.log"
Jul 14 12:05:41.098748 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:41.098721    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-ld599_e223985c-991b-4f3e-836f-8fbd992e4dea/manager/0.log"
Jul 14 12:05:41.971184 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:41.971155    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733825-r28lw_7b222f5d-91b7-455a-8eb1-422d69461b12/cleanup/0.log"
Jul 14 12:05:42.193584 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:42.193552    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-4b103af8-tmdz2_2b32a5ce-ce3a-4801-a7ee-932b0df3664f/revoke-keys/0.log"
Jul 14 12:05:42.421459 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:42.421430    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-565f5c-w5hh9_477fdf7b-b134-4a9d-a6d2-f5ade4af2865/revoke-keys/0.log"
Jul 14 12:05:42.534189 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:42.534159    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-290745-nc7ck_54044d7c-b933-4210-a26a-a604ff2300a7/revoke-keys/0.log"
Jul 14 12:05:42.655632 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:42.655608    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-b8cbc777-prrt7_5711c788-ff33-4fb4-9bab-949dd3498af4/revoke-keys/0.log"
Jul 14 12:05:42.777097 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:42.777069    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-8567a3-65bp9_aee2cd1b-8389-4fa4-a6eb-d8a430fa3013/revoke-keys/0.log"
Jul 14 12:05:42.896692 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:42.896661    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-738ce7-89nnb_fff35806-07ce-4530-9d2d-24d1162228af/revoke-keys/0.log"
Jul 14 12:05:43.010597 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:43.010566    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-f22ceb3a-7zjzf_8976e84c-056f-47b0-885b-c7be26f436b5/revoke-keys/0.log"
Jul 14 12:05:43.121371 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:43.121301    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-3f0f93e1-9qtv9_dd6d0847-3fde-4fd3-8331-e818c3d5e648/revoke-keys/0.log"
Jul 14 12:05:43.237762 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:43.237733    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-67b298da-2dhxl_a434d7d4-345a-440f-8af0-03604a710d3d/revoke-keys/0.log"
Jul 14 12:05:43.374147 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:43.374074    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-1cc796-5hfj4_41c54d39-f9fb-40f5-9056-afd63d7c6dc3/revoke-keys/0.log"
Jul 14 12:05:43.487624 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:43.487600    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-0508a3-jgh4v_9b703eed-bb82-4ffc-9ddf-9e51bf31458c/revoke-keys/0.log"
Jul 14 12:05:44.935364 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:44.935335    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-97f67d9-g9wmz_52f31154-5875-44bc-86e9-514da0d7db6f/authorino/0.log"
Jul 14 12:05:45.184611 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:45.184564    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qj8v7_85c9751d-61bd-4072-b205-09b87b3ed292/manager/0.log"
Jul 14 12:05:45.299052 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:45.299028    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-td7qd_94f17e02-1ba9-4eae-aea7-fd0c2596c6f5/kuadrant-console-plugin/0.log"
Jul 14 12:05:45.672973 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:45.672886    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b6pjt_e2847ace-c277-499f-83e2-6e3e24e83cdb/limitador/0.log"
Jul 14 12:05:46.571133 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:46.571101    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-p5kdl_949e0818-e7ee-42df-9b5e-663f5527e81b/kube-auth-proxy/0.log"
Jul 14 12:05:47.949608 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:47.949575    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-pknt9_505e78a2-7e28-4c92-b70d-262066e2e270/main/0.log"
Jul 14 12:05:51.874443 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.874411    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5q25/must-gather-dn74j"]
Jul 14 12:05:51.874834 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.874753    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b703eed-bb82-4ffc-9ddf-9e51bf31458c" containerName="revoke-keys"
Jul 14 12:05:51.874834 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.874764    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b703eed-bb82-4ffc-9ddf-9e51bf31458c" containerName="revoke-keys"
Jul 14 12:05:51.874910 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.874838    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b703eed-bb82-4ffc-9ddf-9e51bf31458c" containerName="revoke-keys"
Jul 14 12:05:51.877958 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.877935    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:51.880419 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.880397    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5q25\"/\"openshift-service-ca.crt\""
Jul 14 12:05:51.880514 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.880437    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5q25\"/\"kube-root-ca.crt\""
Jul 14 12:05:51.890270 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.890239    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/must-gather-dn74j"]
Jul 14 12:05:51.931932 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.931899    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjrrz\" (UniqueName: \"kubernetes.io/projected/beaa065d-a4b1-428c-bc60-3d7024a376bd-kube-api-access-qjrrz\") pod \"must-gather-dn74j\" (UID: \"beaa065d-a4b1-428c-bc60-3d7024a376bd\") " pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:51.931932 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:51.931941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/beaa065d-a4b1-428c-bc60-3d7024a376bd-must-gather-output\") pod \"must-gather-dn74j\" (UID: \"beaa065d-a4b1-428c-bc60-3d7024a376bd\") " pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:52.033384 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:52.033349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/beaa065d-a4b1-428c-bc60-3d7024a376bd-must-gather-output\") pod \"must-gather-dn74j\" (UID: \"beaa065d-a4b1-428c-bc60-3d7024a376bd\") " pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:52.033569 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:52.033477    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjrrz\" (UniqueName: \"kubernetes.io/projected/beaa065d-a4b1-428c-bc60-3d7024a376bd-kube-api-access-qjrrz\") pod \"must-gather-dn74j\" (UID: \"beaa065d-a4b1-428c-bc60-3d7024a376bd\") " pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:52.033843 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:52.033817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/beaa065d-a4b1-428c-bc60-3d7024a376bd-must-gather-output\") pod \"must-gather-dn74j\" (UID: \"beaa065d-a4b1-428c-bc60-3d7024a376bd\") " pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:52.042518 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:52.042496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjrrz\" (UniqueName: \"kubernetes.io/projected/beaa065d-a4b1-428c-bc60-3d7024a376bd-kube-api-access-qjrrz\") pod \"must-gather-dn74j\" (UID: \"beaa065d-a4b1-428c-bc60-3d7024a376bd\") " pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:52.187853 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:52.187765    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/must-gather-dn74j"
Jul 14 12:05:52.324123 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:52.324093    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/must-gather-dn74j"]
Jul 14 12:05:52.326076 ip-10-0-142-172 kubenswrapper[2564]: W0714 12:05:52.326042    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbeaa065d_a4b1_428c_bc60_3d7024a376bd.slice/crio-37d8d06f39d109e5c5c3ccbef724a35c2fdcfb5600866c740fe45cc66a50e39f WatchSource:0}: Error finding container 37d8d06f39d109e5c5c3ccbef724a35c2fdcfb5600866c740fe45cc66a50e39f: Status 404 returned error can't find the container with id 37d8d06f39d109e5c5c3ccbef724a35c2fdcfb5600866c740fe45cc66a50e39f
Jul 14 12:05:52.959789 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:52.959752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/must-gather-dn74j" event={"ID":"beaa065d-a4b1-428c-bc60-3d7024a376bd","Type":"ContainerStarted","Data":"37d8d06f39d109e5c5c3ccbef724a35c2fdcfb5600866c740fe45cc66a50e39f"}
Jul 14 12:05:53.969506 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:53.969459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/must-gather-dn74j" event={"ID":"beaa065d-a4b1-428c-bc60-3d7024a376bd","Type":"ContainerStarted","Data":"58e05dee50fb7e56c8a93551dd0ae46ba032293d3dd0785bc17055e667c1a497"}
Jul 14 12:05:53.970013 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:53.969526    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/must-gather-dn74j" event={"ID":"beaa065d-a4b1-428c-bc60-3d7024a376bd","Type":"ContainerStarted","Data":"5a8fd6d0a27e58abe0d4259568edcbeb0a71a66a6db640623807e796e26ca0e7"}
Jul 14 12:05:53.988297 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:53.988231    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5q25/must-gather-dn74j" podStartSLOduration=2.104513758 podStartE2EDuration="2.988210184s" totalImagesPullingTime="883.696426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 12:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:05:53.986759653 +0000 UTC m=+2847.787315392" watchObservedRunningTime="2026-07-14 12:05:53.988210184 +0000 UTC m=+2847.788765926"
Jul 14 12:05:54.957584 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:54.957552    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cbj6p_e0ffcf77-e340-4c18-96d2-c069f045d964/global-pull-secret-syncer/0.log"
Jul 14 12:05:55.089882 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:55.089853    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kl6lg_19adb3db-cb83-433e-bf71-997f74e2c60f/konnectivity-agent/0.log"
Jul 14 12:05:55.169947 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:05:55.169917    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-172.ec2.internal_bdebd314971aba3dd18c52401ee8209e/haproxy/0.log"
Jul 14 12:06:00.212118 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:00.210591    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-97f67d9-g9wmz_52f31154-5875-44bc-86e9-514da0d7db6f/authorino/0.log"
Jul 14 12:06:00.288494 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:00.288463    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qj8v7_85c9751d-61bd-4072-b205-09b87b3ed292/manager/0.log"
Jul 14 12:06:00.326058 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:00.326019    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-td7qd_94f17e02-1ba9-4eae-aea7-fd0c2596c6f5/kuadrant-console-plugin/0.log"
Jul 14 12:06:00.551676 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:00.551644    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b6pjt_e2847ace-c277-499f-83e2-6e3e24e83cdb/limitador/0.log"
Jul 14 12:06:02.728292 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:02.728253    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-m7hz6_54d87f2b-e0e6-4d75-a261-db1b140eac5e/monitoring-plugin/0.log"
Jul 14 12:06:02.937562 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:02.937473    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swcvq_fb1dd50f-b737-4e06-b6d3-325405b290a8/node-exporter/0.log"
Jul 14 12:06:02.960281 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:02.960254    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swcvq_fb1dd50f-b737-4e06-b6d3-325405b290a8/kube-rbac-proxy/0.log"
Jul 14 12:06:02.983987 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:02.983954    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-swcvq_fb1dd50f-b737-4e06-b6d3-325405b290a8/init-textfile/0.log"
Jul 14 12:06:03.019205 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.019159    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tjpkp_cc2a5099-5319-4720-85db-4c54686b4293/kube-rbac-proxy-main/0.log"
Jul 14 12:06:03.043868 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.043828    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tjpkp_cc2a5099-5319-4720-85db-4c54686b4293/kube-rbac-proxy-self/0.log"
Jul 14 12:06:03.084373 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.084337    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-tjpkp_cc2a5099-5319-4720-85db-4c54686b4293/openshift-state-metrics/0.log"
Jul 14 12:06:03.120691 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.120664    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7039a24-5b18-4989-a851-617b918efdd9/prometheus/0.log"
Jul 14 12:06:03.147203 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.147176    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7039a24-5b18-4989-a851-617b918efdd9/config-reloader/0.log"
Jul 14 12:06:03.174021 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.173991    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7039a24-5b18-4989-a851-617b918efdd9/thanos-sidecar/0.log"
Jul 14 12:06:03.197648 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.197574    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7039a24-5b18-4989-a851-617b918efdd9/kube-rbac-proxy-web/0.log"
Jul 14 12:06:03.225803 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.225757    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7039a24-5b18-4989-a851-617b918efdd9/kube-rbac-proxy/0.log"
Jul 14 12:06:03.256085 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.256055    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7039a24-5b18-4989-a851-617b918efdd9/kube-rbac-proxy-thanos/0.log"
Jul 14 12:06:03.283303 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.283274    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7039a24-5b18-4989-a851-617b918efdd9/init-config-reloader/0.log"
Jul 14 12:06:03.363578 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.363538    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-7p5z4_67309b0e-d8d1-4b17-83f9-d7712eede2b0/prometheus-operator-admission-webhook/0.log"
Jul 14 12:06:03.481610 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.481535    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/thanos-query/0.log"
Jul 14 12:06:03.506413 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.506384    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy-web/0.log"
Jul 14 12:06:03.532713 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.532676    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy/0.log"
Jul 14 12:06:03.563548 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.563513    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/prom-label-proxy/0.log"
Jul 14 12:06:03.589557 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.589527    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy-rules/0.log"
Jul 14 12:06:03.614482 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:03.614447    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545bdb9697-c68hp_c97234cd-3afe-47fc-85cd-be69ea9b3305/kube-rbac-proxy-metrics/0.log"
Jul 14 12:06:05.040459 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.040420    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"]
Jul 14 12:06:05.046434 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.046397    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.048448 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.048414    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k5q25\"/\"default-dockercfg-m6pb5\""
Jul 14 12:06:05.055640 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.055603    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"]
Jul 14 12:06:05.167497 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.167458    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-podres\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.167802 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.167780    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-proc\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.167978 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.167963    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-sys\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.168144 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.168127    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4pb\" (UniqueName: \"kubernetes.io/projected/752ae9fd-d31e-46e7-862a-6cc80c82b752-kube-api-access-6v4pb\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.168329 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.168315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-lib-modules\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.270893 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.270853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-proc\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271085 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.270928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-sys\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271085 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.270981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4pb\" (UniqueName: \"kubernetes.io/projected/752ae9fd-d31e-46e7-862a-6cc80c82b752-kube-api-access-6v4pb\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271085 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.270988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-proc\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271085 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.271019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-sys\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271085 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.271067    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-lib-modules\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271382 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.271120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-podres\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271382 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.271223    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-lib-modules\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.271382 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.271242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/752ae9fd-d31e-46e7-862a-6cc80c82b752-podres\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.279675 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.279645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4pb\" (UniqueName: \"kubernetes.io/projected/752ae9fd-d31e-46e7-862a-6cc80c82b752-kube-api-access-6v4pb\") pod \"perf-node-gather-daemonset-8tp85\" (UID: \"752ae9fd-d31e-46e7-862a-6cc80c82b752\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.361087 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.360968    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:05.520032 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:05.519925    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"]
Jul 14 12:06:06.063310 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:06.063267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85" event={"ID":"752ae9fd-d31e-46e7-862a-6cc80c82b752","Type":"ContainerStarted","Data":"19f57da006b408e6de4870c3a244aec43675e434673416c7877c31e8c7f8b831"}
Jul 14 12:06:06.063808 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:06.063333    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85" event={"ID":"752ae9fd-d31e-46e7-862a-6cc80c82b752","Type":"ContainerStarted","Data":"a67b38e82444450da1214e2d4fe91240b1dd279297f721eb51db5eb0d003fe8b"}
Jul 14 12:06:06.064144 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:06.064120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:06.083868 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:06.083801    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85" podStartSLOduration=1.083778043 podStartE2EDuration="1.083778043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 12:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:06:06.080677502 +0000 UTC m=+2859.881233238" watchObservedRunningTime="2026-07-14 12:06:06.083778043 +0000 UTC m=+2859.884333784"
Jul 14 12:06:07.755688 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:07.755663    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z5f9m_aeadd9fa-ef3b-40f2-b2f7-e69191293b3b/dns/0.log"
Jul 14 12:06:07.778797 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:07.778769    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z5f9m_aeadd9fa-ef3b-40f2-b2f7-e69191293b3b/kube-rbac-proxy/0.log"
Jul 14 12:06:07.805367 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:07.805339    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dgrpv_531c861d-fdf2-4713-942d-c18700ba8aa5/dns-node-resolver/0.log"
Jul 14 12:06:08.408173 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:08.408140    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b66646578-mlbpl_85e379da-4380-4406-9f31-f9730d9ffd46/registry/0.log"
Jul 14 12:06:08.431237 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:08.431212    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bl2sn_c3ca13a8-f8ad-4565-b6f9-f3a5342e3998/node-ca/0.log"
Jul 14 12:06:09.688929 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:09.688895    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-p5kdl_949e0818-e7ee-42df-9b5e-663f5527e81b/kube-auth-proxy/0.log"
Jul 14 12:06:10.655995 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:10.655964    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5wpn6_813f9693-f3e7-4c38-8ed1-7be66382f4ef/serve-healthcheck-canary/0.log"
Jul 14 12:06:11.285436 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:11.285405    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q78dz_08dd0d2a-d514-4f3b-b980-b8dfdf55e50f/kube-rbac-proxy/0.log"
Jul 14 12:06:11.309641 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:11.309611    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q78dz_08dd0d2a-d514-4f3b-b980-b8dfdf55e50f/exporter/0.log"
Jul 14 12:06:11.336227 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:11.336200    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q78dz_08dd0d2a-d514-4f3b-b980-b8dfdf55e50f/extractor/0.log"
Jul 14 12:06:13.090671 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:13.090641    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-8tp85"
Jul 14 12:06:14.379965 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:14.379850    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-ld599_e223985c-991b-4f3e-836f-8fbd992e4dea/manager/0.log"
Jul 14 12:06:14.409569 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:14.409538    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-ld599_e223985c-991b-4f3e-836f-8fbd992e4dea/copy-manifests/0.log"
Jul 14 12:06:15.350031 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:15.350001    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-p5kdl_949e0818-e7ee-42df-9b5e-663f5527e81b/kube-auth-proxy/0.log"
Jul 14 12:06:15.873837 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:15.873807    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-985446bdb-8hzx9_243b252f-6f7c-4aac-afc8-180b8e7171fa/manager/0.log"
Jul 14 12:06:15.934196 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:15.934167    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-d52zh_2d25d998-862d-426d-916b-fedbf8cc0cdd/openshift-lws-operator/0.log"
Jul 14 12:06:16.870487 ip-10-0-142-172 kubenswrapper[2564]: I0714 12:06:16.870417    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76f9775b55-p5kdl_949e0818-e7ee-42df-9b5e-663f5527e81b/kube-auth-proxy/0.log"