Jul 06 14:00:46.135227 ip-10-0-129-92 systemd[1]: Starting Kubernetes Kubelet... Jul 06 14:00:49.128060 ip-10-0-129-92 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:00:49.128060 ip-10-0-129-92 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 14:00:49.128060 ip-10-0-129-92 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:00:49.128060 ip-10-0-129-92 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 14:00:49.128060 ip-10-0-129-92 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:00:49.129314 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.129224 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 14:00:49.133166 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133148 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:00:49.133166 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133165 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133169 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133172 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133177 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133182 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133185 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133188 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133191 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133194 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133197 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133201 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133204 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133207 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133210 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133213 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133216 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133218 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133221 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133224 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133231 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:00:49.133232 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133234 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133237 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133241 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133243 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133246 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133249 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133252 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133254 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133257 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133260 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133263 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133267 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133271 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133274 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133276 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133279 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133281 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133284 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133287 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:00:49.133688 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133289 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133293 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133295 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133298 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133300 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133303 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133306 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133308 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133311 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133313 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133318 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133321 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133324 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133327 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133329 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133334 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133337 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133339 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133342 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:00:49.134182 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133345 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133348 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133352 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133354 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133357 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133360 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133362 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133365 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133368 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133370 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133373 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133375 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133381 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133384 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133386 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133389 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133392 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133394 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133396 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133399 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:00:49.134648 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133401 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133404 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133407 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133409 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133412 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133414 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133417 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133419 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133422 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133425 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133428 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133431 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133433 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133436 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133438 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133441 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133443 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133446 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133449 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.133451 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:00:49.135117 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134163 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134169 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134172 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134175 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134178 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134181 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134184 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134186 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134189 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134192 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134195 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134197 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134203 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134206 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134209 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134212 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134215 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134217 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134220 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:00:49.135570 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134222 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134226 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134230 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134233 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134235 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134238 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134240 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134243 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134246 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134248 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134251 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134254 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134256 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134259 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134261 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134264 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134266 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134271 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134274 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:00:49.136073 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134278 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134281 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134284 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134286 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134289 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134291 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134294 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134297 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134301 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134304 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134307 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134310 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134312 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134315 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134318 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134321 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134324 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134326 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134329 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134331 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:00:49.136557 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134334 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134337 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134339 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134342 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134344 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134347 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134349 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134352 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134354 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134357 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134359 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134362 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134364 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134367 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134370 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134372 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134375 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134378 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134380 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134383 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:00:49.137067 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134385 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134388 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134390 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134393 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134396 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134398 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134401 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134405 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134408 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134411 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134413 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134416 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134419 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134422 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134424 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134427 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134429 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134432 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134434 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:00:49.137548 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134437 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134439 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134804 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134808 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134811 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134813 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134817 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134820 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134822 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134826 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134828 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134831 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134834 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134837 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134840 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134842 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134845 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134847 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134850 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134853 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:00:49.138033 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134855 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134860 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134864 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134867 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134870 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134874 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134876 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134879 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134882 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134885 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134888 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134890 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134893 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134912 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134916 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134919 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134921 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134924 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134926 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:00:49.138506 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134930 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134932 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134935 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134937 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134940 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134942 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134945 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134949 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134951 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134954 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134957 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134960 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134962 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134965 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134967 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134970 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134973 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134975 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134978 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134980 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:00:49.138987 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134983 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134985 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134988 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134991 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134994 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.134996 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135002 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135004 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135007 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135011 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135014 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135017 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135020 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135023 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135026 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135029 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135032 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135035 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135037 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:00:49.139450 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135040 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135043 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135046 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135049 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135051 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135054 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135057 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135059 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135062 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135065 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135068 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135070 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135073 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135075 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135078 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135080 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135083 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135085 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135088 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135092 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:00:49.139922 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135095 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135097 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135100 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135170 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135177 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135184 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135189 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135194 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135197 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135202 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135206 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135210 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135213 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135216 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135220 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135223 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135227 2562 flags.go:64] FLAG: --cgroup-root="" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135230 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135233 2562 flags.go:64] FLAG: --client-ca-file="" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135235 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135238 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135243 2562 flags.go:64] FLAG: --cluster-domain="" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135246 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135249 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 14:00:49.140399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135252 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135255 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135259 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135262 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135265 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135269 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135272 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135276 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135280 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135283 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135286 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135290 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135293 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135296 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135299 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135302 2562 flags.go:64] FLAG: --enable-server="true" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135306 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135311 2562 flags.go:64] FLAG: --event-burst="100" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135314 2562 flags.go:64] FLAG: --event-qps="50" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135317 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135320 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135323 2562 flags.go:64] FLAG: --eviction-hard="" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135328 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135331 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135334 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 14:00:49.140990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135337 2562 flags.go:64] FLAG: --eviction-soft="" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135340 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135343 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135346 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135349 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135352 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135356 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135358 2562 flags.go:64] FLAG: --feature-gates="" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135362 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135365 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135368 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135372 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135375 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135378 2562 flags.go:64] FLAG: --help="false" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135382 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135385 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135388 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135391 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135395 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135399 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135402 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135405 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135408 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 14:00:49.141568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135411 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135414 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135417 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135420 2562 flags.go:64] FLAG: --kube-reserved="" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135423 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135426 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135430 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135433 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135436 2562 flags.go:64] FLAG: --lock-file="" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135439 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135442 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135445 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135450 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135453 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135456 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135459 2562 flags.go:64] FLAG: --logging-format="text" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135463 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135466 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135469 2562 flags.go:64] FLAG: --manifest-url="" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135472 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135476 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135479 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135484 2562 flags.go:64] FLAG: --max-pods="110" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135488 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135491 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 14:00:49.142218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135494 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135497 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135501 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135504 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135507 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135514 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135518 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135521 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135524 2562 flags.go:64] FLAG: --pod-cidr="" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135527 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135533 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135536 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135539 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135542 2562 flags.go:64] FLAG: --port="10250" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135545 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135548 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0db03f60da7e6351d" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135551 2562 flags.go:64] FLAG: --qos-reserved="" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135554 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135557 2562 flags.go:64] FLAG: --register-node="true" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135560 2562 flags.go:64] FLAG: --register-with-taints="" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135564 2562 flags.go:64] FLAG: --registry-burst="10" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135567 2562 flags.go:64] FLAG: --registry-qps="5" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135570 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135573 2562 flags.go:64] FLAG: --reserved-memory="" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135577 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 14:00:49.142985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135580 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135583 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135586 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135589 2562 flags.go:64] FLAG: --runonce="false" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135592 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135597 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135600 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135602 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135606 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135609 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135612 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135615 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135618 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135622 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135625 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135628 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135631 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135634 2562 flags.go:64] FLAG: --system-cgroups="" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135637 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135643 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135646 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135650 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135655 2562 flags.go:64] FLAG: --tls-min-version="" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135658 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135661 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 14:00:49.144029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135664 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135667 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135670 2562 flags.go:64] FLAG: --v="2" Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135679 2562 flags.go:64] FLAG: --version="false" Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135683 2562 flags.go:64] FLAG: --vmodule="" Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135686 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.135690 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135831 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135835 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135838 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135841 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135844 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135850 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135853 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135856 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135860 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135862 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135865 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135868 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135871 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:00:49.145042 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135874 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135878 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135882 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135885 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135888 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135892 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135908 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135911 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135915 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135918 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135921 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135924 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135927 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135929 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135932 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135935 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135937 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135940 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135942 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:00:49.145528 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135945 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135947 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135950 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135953 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135956 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135960 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135962 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135965 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135968 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135971 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135973 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135976 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135978 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135981 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135984 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135986 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135988 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135991 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135994 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.135997 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:00:49.146198 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136000 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136003 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136006 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136009 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136011 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136014 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136017 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136019 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136022 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136024 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136027 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136029 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136032 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136035 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136037 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136040 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136043 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136047 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136050 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136052 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:00:49.147119 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136055 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136058 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136060 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136063 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136066 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136068 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136071 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136074 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136076 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136079 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136082 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136084 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136087 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136091 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136094 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136097 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136099 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136102 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136106 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136110 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:00:49.147694 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136113 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136116 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136119 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136121 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136124 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136127 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.136129 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.137172 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.144860 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.144882 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.145765 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.145778 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145869 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145878 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145884 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:00:49.148498 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145889 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145894 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145915 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145920 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145924 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145929 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145933 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145937 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145941 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145945 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145950 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145954 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145957 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145962 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145966 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145970 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145974 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145979 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145984 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145988 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:00:49.149158 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145992 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.145996 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146001 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146005 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146010 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146015 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146021 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146025 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146029 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146033 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146037 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146041 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146046 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146050 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146054 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146058 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146063 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146067 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146074 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:00:49.149672 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146080 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146086 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146092 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146097 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146101 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146106 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146110 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146115 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146119 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146124 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146128 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146132 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146136 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146140 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146144 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146148 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146152 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146156 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146161 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146166 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:00:49.150327 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146171 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146175 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146179 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146184 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146188 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146192 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146196 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146200 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146204 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146208 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146211 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146215 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146219 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146223 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146227 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146232 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146236 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146241 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146245 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146249 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:00:49.150838 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146254 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146258 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146262 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146267 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146271 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146274 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146279 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146286 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146293 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146300 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146305 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146310 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146315 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146320 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146325 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146331 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:00:49.151334 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146335 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.146343 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146532 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146540 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146545 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146549 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146554 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146558 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146565 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146571 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146575 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146580 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146584 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146589 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146593 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:00:49.151714 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146597 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146602 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146606 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146610 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146615 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146619 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146623 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146628 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146633 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146637 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146641 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146645 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146650 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146655 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146660 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146669 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146673 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146678 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146682 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146686 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:00:49.152157 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146690 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146694 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146699 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146703 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146707 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146711 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146715 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146720 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146724 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146730 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146736 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146741 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146745 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146750 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146754 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146758 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146762 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146766 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146770 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:00:49.152633 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146774 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146779 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146783 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146787 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146792 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146796 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146801 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146805 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146809 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146815 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146819 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146823 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146827 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146834 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146840 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146845 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146849 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146853 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146858 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146862 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:00:49.153180 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146867 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146871 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146876 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146880 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146884 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146888 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146892 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146912 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146917 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146921 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146925 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146929 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146935 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146940 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146945 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146949 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146954 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146958 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146962 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146966 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:00:49.153665 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146971 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146975 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146980 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146984 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146989 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146993 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:49.146997 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.147005 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.147176 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.149888 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.151826 2562 server.go:1019] "Starting client certificate rotation" Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.151929 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:00:49.154238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.152733 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:00:49.155320 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.155302 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:00:49.161478 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.161454 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:00:49.170219 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.170197 2562 log.go:25] "Validated CRI v1 runtime API" Jul 06 14:00:49.172247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.172234 2562 log.go:25] "Validated CRI v1 image API" Jul 06 14:00:49.173624 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.173609 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 14:00:49.176214 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.176187 2562 fs.go:135] Filesystem UUIDs: map[10c62d13-e5d0-466e-a72d-8c10ce67d7eb:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a72b6132-bef9-4f32-a263-6cc34bfdc1a2:/dev/nvme0n1p4] Jul 06 14:00:49.176281 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.176212 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 14:00:49.181427 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.181319 2562 manager.go:216] Machine: {Timestamp:2026-07-06 14:00:49.17993636 +0000 UTC m=+0.368985418 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099078 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24b14063e485ad58826e873855fcd4 SystemUUID:ec24b140-63e4-85ad-5882-6e873855fcd4 BootID:fca6b6ff-0d3f-4139-bb83-c8f410002941 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d5:fb:55:bb:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d5:fb:55:bb:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:e3:86:7b:e7:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 14:00:49.181427 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.181420 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 14:00:49.181551 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.181507 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 14:00:49.183044 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.183026 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:00:49.187360 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.187332 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 14:00:49.187513 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.187363 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-92.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 14:00:49.188960 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.188949 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 14:00:49.188996 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.188962 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 14:00:49.188996 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.188970 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:00:49.189794 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.189785 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:00:49.191224 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.191214 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 14:00:49.195001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.194989 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:00:49.195112 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195103 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 14:00:49.195150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195134 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 06 14:00:49.195150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195144 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 14:00:49.195200 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195158 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 14:00:49.195200 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195168 2562 kubelet.go:398] "Adding apiserver pod source" Jul 06 14:00:49.195200 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195176 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 14:00:49.195586 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195573 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:00:49.195627 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.195602 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:00:49.198040 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.198016 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 14:00:49.200501 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.200487 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 14:00:49.200557 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.200517 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 14:00:49.201790 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201779 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 14:00:49.201827 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201795 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 14:00:49.201827 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201801 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 14:00:49.201827 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201806 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 14:00:49.201827 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201814 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 14:00:49.201827 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201822 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 14:00:49.201972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201832 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 14:00:49.201972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201837 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 14:00:49.201972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201850 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 14:00:49.201972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201855 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 14:00:49.201972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201863 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 14:00:49.201972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.201868 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 14:00:49.204183 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.204164 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 14:00:49.204259 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.204190 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 14:00:49.207245 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.207193 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-92.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 14:00:49.207245 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.207220 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 14:00:49.207530 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.207511 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 14:00:49.208294 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.208277 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n5xhp" Jul 06 14:00:49.208452 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.208440 2562 server.go:1268] "Started kubelet" Jul 06 14:00:49.208534 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.208506 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 14:00:49.208882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.208845 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 14:00:49.208976 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.208922 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 14:00:49.209433 ip-10-0-129-92 systemd[1]: Started Kubernetes Kubelet. Jul 06 14:00:49.210637 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.210625 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 06 14:00:49.211346 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.211329 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 14:00:49.214135 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.214108 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 14:00:49.216357 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.216343 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n5xhp" Jul 06 14:00:49.217154 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.217138 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 14:00:49.217154 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.217141 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 14:00:49.217337 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.217316 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.217775 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.217758 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 14:00:49.217775 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.217760 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 14:00:49.217922 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.217786 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 14:00:49.218450 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218155 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 06 14:00:49.218450 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218168 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 06 14:00:49.218450 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218283 2562 factory.go:55] Registering systemd factory Jul 06 14:00:49.218450 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218296 2562 factory.go:223] Registration of the systemd container factory successfully Jul 06 14:00:49.218773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218550 2562 factory.go:153] Registering CRI-O factory Jul 06 14:00:49.218773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218566 2562 factory.go:223] Registration of the crio container factory successfully Jul 06 14:00:49.218773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218632 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 14:00:49.218773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218667 2562 factory.go:103] Registering Raw factory Jul 06 14:00:49.218773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.218683 2562 manager.go:1196] Started watching for new ooms in manager Jul 06 14:00:49.219417 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.219392 2562 manager.go:349] Starting recovery of all containers Jul 06 14:00:49.223133 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.223113 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:00:49.226157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.226105 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 14:00:49.226859 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.226840 2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-92.ec2.internal\" not found" node="ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.234218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.234025 2562 manager.go:354] Recovery completed Jul 06 14:00:49.238873 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.238857 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:00:49.240748 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.240731 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:00:49.240807 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.240769 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:00:49.240807 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.240784 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:00:49.241373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.241359 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 14:00:49.241373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.241365 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:00:49.241373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.241370 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 14:00:49.241517 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.241386 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:00:49.242261 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.242244 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:00:49.242337 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.242272 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:00:49.242337 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.242284 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:00:49.242876 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.242860 2562 policy_none.go:49] "None policy: Start" Jul 06 14:00:49.243221 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.243207 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 14:00:49.243424 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.243408 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 14:00:49.244624 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.244610 2562 policy_none.go:47] "Start" Jul 06 14:00:49.277397 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.277383 2562 manager.go:337] "Starting Device Plugin manager" Jul 06 14:00:49.277484 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.277424 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 14:00:49.277484 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.277436 2562 server.go:87] "Starting device plugin registration server" Jul 06 14:00:49.277699 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.277686 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 14:00:49.277733 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.277700 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 14:00:49.277883 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.277865 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 14:00:49.277997 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.277981 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 14:00:49.277997 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.277994 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 14:00:49.278503 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.278479 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 14:00:49.278584 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.278543 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.378414 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.378346 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:00:49.379535 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.379276 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:00:49.379535 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.379310 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:00:49.379535 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.379320 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:00:49.379535 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.379348 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.382345 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.382307 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 14:00:49.382444 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.382361 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 14:00:49.382444 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.382383 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 14:00:49.382444 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.382412 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 14:00:49.384381 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.384362 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:00:49.391222 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.391202 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.391314 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.391227 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-92.ec2.internal\": node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.453587 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.453560 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.482650 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.482617 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal"] Jul 06 14:00:49.482725 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.482712 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:00:49.485008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.484991 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:00:49.485110 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.485021 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:00:49.485110 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.485033 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:00:49.486424 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.486410 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:00:49.486551 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.486536 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.486613 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.486574 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:00:49.487191 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.487169 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:00:49.487304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.487169 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:00:49.487304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.487199 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:00:49.487304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.487228 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:00:49.487304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.487237 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:00:49.487304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.487253 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:00:49.488382 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.488364 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.488485 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.488394 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:00:49.489805 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.489776 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:00:49.489889 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.489809 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:00:49.489889 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.489824 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:00:49.511617 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.511589 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-92.ec2.internal\" not found" node="ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.516082 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.516063 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-92.ec2.internal\" not found" node="ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.520285 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.520267 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d9eb1da208d4c1c520252bceb70d657-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal\" (UID: \"7d9eb1da208d4c1c520252bceb70d657\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.520344 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.520292 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d9eb1da208d4c1c520252bceb70d657-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal\" (UID: \"7d9eb1da208d4c1c520252bceb70d657\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.520344 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.520311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1524ac3c95d4a1a172ad30214b4048df-config\") pod \"kube-apiserver-proxy-ip-10-0-129-92.ec2.internal\" (UID: \"1524ac3c95d4a1a172ad30214b4048df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.553995 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.553968 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.620706 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.620677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d9eb1da208d4c1c520252bceb70d657-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal\" (UID: \"7d9eb1da208d4c1c520252bceb70d657\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.620829 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.620709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d9eb1da208d4c1c520252bceb70d657-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal\" (UID: \"7d9eb1da208d4c1c520252bceb70d657\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.620829 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.620727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1524ac3c95d4a1a172ad30214b4048df-config\") pod \"kube-apiserver-proxy-ip-10-0-129-92.ec2.internal\" (UID: \"1524ac3c95d4a1a172ad30214b4048df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.620829 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.620768 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d9eb1da208d4c1c520252bceb70d657-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal\" (UID: \"7d9eb1da208d4c1c520252bceb70d657\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.620829 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.620793 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d9eb1da208d4c1c520252bceb70d657-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal\" (UID: \"7d9eb1da208d4c1c520252bceb70d657\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.620829 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.620780 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1524ac3c95d4a1a172ad30214b4048df-config\") pod \"kube-apiserver-proxy-ip-10-0-129-92.ec2.internal\" (UID: \"1524ac3c95d4a1a172ad30214b4048df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.654824 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.654767 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.755490 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.755454 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.814625 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.814581 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.819207 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:49.818855 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" Jul 06 14:00:49.855790 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.855758 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:49.956376 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:49.956280 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:50.056779 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:50.056746 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:50.151300 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.151265 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 14:00:50.151877 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.151466 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:00:50.151877 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.151466 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:00:50.157555 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:50.157530 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-92.ec2.internal\" not found" Jul 06 14:00:50.219287 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.219198 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 13:55:49 +0000 UTC" deadline="2028-03-21 03:23:16.310250758 +0000 UTC" Jul 06 14:00:50.219287 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.219236 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14965h22m26.091019308s" Jul 06 14:00:50.227783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.227760 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:00:50.318177 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.318145 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" Jul 06 14:00:50.330164 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.330134 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:00:50.332676 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.332659 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" Jul 06 14:00:50.342395 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.342372 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:00:50.385453 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:50.385424 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1524ac3c95d4a1a172ad30214b4048df.slice/crio-57864963e3b0808ee6ca39f7a60e05c74e1703e46d551b01a237cd83e1955725 WatchSource:0}: Error finding container 57864963e3b0808ee6ca39f7a60e05c74e1703e46d551b01a237cd83e1955725: Status 404 returned error can't find the container with id 57864963e3b0808ee6ca39f7a60e05c74e1703e46d551b01a237cd83e1955725 Jul 06 14:00:50.389608 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.389593 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:00:50.687124 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:50.687034 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:00:50.787858 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:50.787829 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9eb1da208d4c1c520252bceb70d657.slice/crio-fd92e5286927dfa015fae4b525e523d2bace16916d7cb7a6b1549360f033143b WatchSource:0}: Error finding container fd92e5286927dfa015fae4b525e523d2bace16916d7cb7a6b1549360f033143b: Status 404 returned error can't find the container with id fd92e5286927dfa015fae4b525e523d2bace16916d7cb7a6b1549360f033143b Jul 06 14:00:51.196420 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.196384 2562 apiserver.go:52] "Watching apiserver" Jul 06 14:00:51.205381 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.205356 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 14:00:51.205732 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.205708 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-tdqwr","kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q6nd6","openshift-dns/node-resolver-7fdpb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal","openshift-multus/multus-additional-cni-plugins-fv7dr","openshift-multus/network-metrics-daemon-ll8js","openshift-network-diagnostics/network-check-target-2rn4s","openshift-ovn-kubernetes/ovnkube-node-vcfvm","kube-system/global-pull-secret-syncer-9g6dg","kube-system/konnectivity-agent-ffc78","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v","openshift-image-registry/node-ca-rvnvx","openshift-multus/multus-drpr8"] Jul 06 14:00:51.207400 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.207378 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:51.207503 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.207477 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:00:51.208538 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.208524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.209861 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.209844 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.211150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.211129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.211495 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.211479 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 14:00:51.211574 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.211514 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 14:00:51.211574 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.211564 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m75xr\"" Jul 06 14:00:51.212416 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.212396 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.212631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.212574 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nrc66\"" Jul 06 14:00:51.212631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.212589 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 14:00:51.212738 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.212722 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:00:51.213696 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.213679 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 14:00:51.213880 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.213865 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 14:00:51.214112 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.214098 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvfqx\"" Jul 06 14:00:51.214523 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.214507 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.214966 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.214944 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 14:00:51.215064 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.214965 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 14:00:51.215064 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.215011 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 14:00:51.215315 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.215222 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 14:00:51.215372 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.215331 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-95bn2\"" Jul 06 14:00:51.215984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.215970 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.217045 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.217027 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sgdrn\"" Jul 06 14:00:51.217143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.217126 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 14:00:51.217198 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.217146 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 14:00:51.217315 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.217299 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.218000 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.217984 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 14:00:51.218338 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.218322 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k9bvw\"" Jul 06 14:00:51.218408 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.218385 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 14:00:51.218516 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.218500 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 14:00:51.218604 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.218584 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 14:00:51.218697 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.218673 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 14:00:51.218806 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.218794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:51.218863 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.218848 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:00:51.221270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.220481 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 14:00:51.221270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.220715 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 14:00:51.221270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.220838 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f52l5\"" Jul 06 14:00:51.221270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.221068 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 14:00:51.221270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.221147 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 14:00:51.221547 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.221296 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 14:00:51.221594 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.221545 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.221639 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.221622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:00:51.222629 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.222610 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.223833 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.223815 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.225270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.225247 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 14:00:51.225362 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.225338 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-89j4r\"" Jul 06 14:00:51.225422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.225412 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:00:51.225477 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.225467 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 14:00:51.226591 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.226568 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 14:00:51.226721 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.226702 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h5gdb\"" Jul 06 14:00:51.226781 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.226722 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 14:00:51.226781 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.226712 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 14:00:51.227445 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.227539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-env-overrides\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.227539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-kubernetes\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.227539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rh88\" (UniqueName: \"kubernetes.io/projected/90093b1d-bd1c-415c-9273-50bc6c7c84dc-kube-api-access-5rh88\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.227539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227530 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9zz\" (UniqueName: \"kubernetes.io/projected/09458f3b-e5f7-4f3d-b621-5396d4176382-kube-api-access-2l9zz\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-sys\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-cni-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227613 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-modprobe-d\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-k8s-cni-cncf-io\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-system-cni-dir\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysctl-conf\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.227703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-host\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227720 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90093b1d-bd1c-415c-9273-50bc6c7c84dc-tmp\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227741 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09688289-2a10-441b-bc83-c7453ea6bb61-serviceca\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227755 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-cni-bin\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-kubelet\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227783 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-run-netns\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-hosts-file\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227825 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b32a7a04-eee4-4d9d-8026-97d8501d6d50-agent-certs\") pod \"konnectivity-agent-ffc78\" (UID: \"b32a7a04-eee4-4d9d-8026-97d8501d6d50\") " pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-system-cni-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227866 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-netns\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-conf-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-node-log\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysctl-d\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-systemd\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.227991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-os-release\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.228104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh992\" (UniqueName: \"kubernetes.io/projected/955a1098-f214-44eb-bcb0-c73ccaba2be5-kube-api-access-gh992\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovn-node-metrics-cert\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228124 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-var-lib-kubelet\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09688289-2a10-441b-bc83-c7453ea6bb61-host\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-slash\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-cni-bin\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228224 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-run\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-lib-modules\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-var-lib-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-cni-netd\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysconfig\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-etc-kubernetes\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228369 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovnkube-config\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228385 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228401 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-cni-binary-copy\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.228858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-socket-dir-parent\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-cni-multus\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228466 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f12f1a7e-9909-4508-b69c-095ffc809462-multus-daemon-config\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-systemd\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-tuned\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-tmp-dir\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-hostroot\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228566 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-kubelet\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-ovn\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-log-socket\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovnkube-script-lib\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns6tk\" (UniqueName: \"kubernetes.io/projected/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-kube-api-access-ns6tk\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b32a7a04-eee4-4d9d-8026-97d8501d6d50-konnectivity-ca\") pod \"konnectivity-agent-ffc78\" (UID: \"b32a7a04-eee4-4d9d-8026-97d8501d6d50\") " pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-multus-certs\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.229631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-os-release\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chcbh\" (UniqueName: \"kubernetes.io/projected/09688289-2a10-441b-bc83-c7453ea6bb61-kube-api-access-chcbh\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f12f1a7e-9909-4508-b69c-095ffc809462-cni-binary-copy\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228833 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5vzc\" (UniqueName: \"kubernetes.io/projected/f12f1a7e-9909-4508-b69c-095ffc809462-kube-api-access-d5vzc\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-cnibin\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228936 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4ljc\" (UniqueName: \"kubernetes.io/projected/aa080866-0ad2-43b4-90b6-68339fdcffc2-kube-api-access-x4ljc\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-cnibin\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.228978 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-systemd-units\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.230386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.229005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-etc-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.235017 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.234996 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:00:51.266831 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.266812 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4xp2" Jul 06 14:00:51.274668 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.274650 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4xp2" Jul 06 14:00:51.314997 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.314965 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:00:51.318808 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.318789 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 14:00:51.329539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysctl-d\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.329605 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-systemd\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.329605 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329561 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:51.329605 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329581 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24b7k\" (UniqueName: \"kubernetes.io/projected/8485d94b-b2ce-4c28-94eb-0e434864be9f-kube-api-access-24b7k\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.329605 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329596 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-os-release\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh992\" (UniqueName: \"kubernetes.io/projected/955a1098-f214-44eb-bcb0-c73ccaba2be5-kube-api-access-gh992\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovn-node-metrics-cert\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329655 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-systemd\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.329706 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysctl-d\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-var-lib-kubelet\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09688289-2a10-441b-bc83-c7453ea6bb61-host\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.329781 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs podName:09458f3b-e5f7-4f3d-b621-5396d4176382 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:51.829759478 +0000 UTC m=+3.018808514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs") pod "network-metrics-daemon-ll8js" (UID: "09458f3b-e5f7-4f3d-b621-5396d4176382") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:51.329783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329787 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09688289-2a10-441b-bc83-c7453ea6bb61-host\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329722 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-os-release\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-slash\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329778 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-var-lib-kubelet\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-cni-bin\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329868 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-slash\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-run\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329911 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329924 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-lib-modules\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329926 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-cni-bin\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-var-lib-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329950 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-run\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-cni-netd\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysconfig\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330029 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-cni-netd\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330031 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-var-lib-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.330238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.329998 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-etc-kubernetes\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330078 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-lib-modules\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysconfig\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330090 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovnkube-config\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330120 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-etc-kubernetes\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330136 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f31c9785-76aa-4397-8383-3d5d06adc6b8-kubelet-config\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330201 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-cni-binary-copy\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-socket-dir-parent\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330310 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-socket-dir-parent\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-cni-multus\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f12f1a7e-9909-4508-b69c-095ffc809462-multus-daemon-config\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-systemd\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-cni-multus\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-tuned\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.331087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-tmp-dir\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330469 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-systemd\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330477 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/683282e2-842c-44fc-be74-df395fa5876b-host-slash\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-hostroot\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-hostroot\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-kubelet\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovnkube-config\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-ovn\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330648 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-run-ovn\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330670 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-kubelet\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330695 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-log-socket\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330736 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-log-socket\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovnkube-script-lib\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330759 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-tmp-dir\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns6tk\" (UniqueName: \"kubernetes.io/projected/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-kube-api-access-ns6tk\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.331875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330821 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b32a7a04-eee4-4d9d-8026-97d8501d6d50-konnectivity-ca\") pod \"konnectivity-agent-ffc78\" (UID: \"b32a7a04-eee4-4d9d-8026-97d8501d6d50\") " pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330823 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-multus-certs\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-os-release\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330883 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-cni-binary-copy\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330951 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-multus-certs\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331015 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-os-release\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.330910 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chcbh\" (UniqueName: \"kubernetes.io/projected/09688289-2a10-441b-bc83-c7453ea6bb61-kube-api-access-chcbh\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stf8r\" (UniqueName: \"kubernetes.io/projected/683282e2-842c-44fc-be74-df395fa5876b-kube-api-access-stf8r\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-socket-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f12f1a7e-9909-4508-b69c-095ffc809462-cni-binary-copy\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5vzc\" (UniqueName: \"kubernetes.io/projected/f12f1a7e-9909-4508-b69c-095ffc809462-kube-api-access-d5vzc\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331167 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f12f1a7e-9909-4508-b69c-095ffc809462-multus-daemon-config\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-cnibin\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331223 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-cnibin\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331339 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4ljc\" (UniqueName: \"kubernetes.io/projected/aa080866-0ad2-43b4-90b6-68339fdcffc2-kube-api-access-x4ljc\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-device-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.332633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-cnibin\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331446 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-cnibin\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-systemd-units\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-etc-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331612 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-systemd-units\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331651 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-registration-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331661 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b32a7a04-eee4-4d9d-8026-97d8501d6d50-konnectivity-ca\") pod \"konnectivity-agent-ffc78\" (UID: \"b32a7a04-eee4-4d9d-8026-97d8501d6d50\") " pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331673 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-etc-openvswitch\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331671 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331716 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f12f1a7e-9909-4508-b69c-095ffc809462-cni-binary-copy\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331744 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-run-ovn-kubernetes\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331753 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovnkube-script-lib\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-env-overrides\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-kubernetes\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rh88\" (UniqueName: \"kubernetes.io/projected/90093b1d-bd1c-415c-9273-50bc6c7c84dc-kube-api-access-5rh88\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.333422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331884 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-kubernetes\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9zz\" (UniqueName: \"kubernetes.io/projected/09458f3b-e5f7-4f3d-b621-5396d4176382-kube-api-access-2l9zz\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-sys\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/683282e2-842c-44fc-be74-df395fa5876b-iptables-alerter-script\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.331991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-sys-fs\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-cni-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-modprobe-d\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-k8s-cni-cncf-io\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-system-cni-dir\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332193 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa080866-0ad2-43b4-90b6-68339fdcffc2-env-overrides\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysctl-conf\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332205 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-cni-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332212 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-sys\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-host\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332277 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-k8s-cni-cncf-io\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-sysctl-conf\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.334269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90093b1d-bd1c-415c-9273-50bc6c7c84dc-tmp\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332736 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09688289-2a10-441b-bc83-c7453ea6bb61-serviceca\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332801 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f31c9785-76aa-4397-8383-3d5d06adc6b8-dbus\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-cni-bin\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332873 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-kubelet\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-run-netns\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332958 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-hosts-file\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.332992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b32a7a04-eee4-4d9d-8026-97d8501d6d50-agent-certs\") pod \"konnectivity-agent-ffc78\" (UID: \"b32a7a04-eee4-4d9d-8026-97d8501d6d50\") " pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-system-cni-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333091 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-netns\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333158 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-cni-bin\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333206 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-var-lib-kubelet\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333261 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-hosts-file\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333261 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/09688289-2a10-441b-bc83-c7453ea6bb61-serviceca\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333262 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-host\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333336 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-system-cni-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333802 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-tuned\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.335103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333964 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90093b1d-bd1c-415c-9273-50bc6c7c84dc-etc-modprobe-d\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.333997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-host-run-netns\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-conf-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-node-log\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334099 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/955a1098-f214-44eb-bcb0-c73ccaba2be5-system-cni-dir\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334160 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-node-log\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334167 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f12f1a7e-9909-4508-b69c-095ffc809462-multus-conf-dir\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334292 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa080866-0ad2-43b4-90b6-68339fdcffc2-host-run-netns\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.334584 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/955a1098-f214-44eb-bcb0-c73ccaba2be5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.335884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.335720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90093b1d-bd1c-415c-9273-50bc6c7c84dc-tmp\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.336335 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.335891 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b32a7a04-eee4-4d9d-8026-97d8501d6d50-agent-certs\") pod \"konnectivity-agent-ffc78\" (UID: \"b32a7a04-eee4-4d9d-8026-97d8501d6d50\") " pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.337459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.337438 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa080866-0ad2-43b4-90b6-68339fdcffc2-ovn-node-metrics-cert\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.340588 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.340570 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:00:51.340588 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.340587 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:00:51.340753 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.340597 2562 projected.go:196] Error preparing data for projected volume kube-api-access-djdj5 for pod openshift-network-diagnostics/network-check-target-2rn4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:51.340753 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.340653 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5 podName:54f69821-e786-41ef-a3bf-41d31c11e2e2 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:51.84064006 +0000 UTC m=+3.029689089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-djdj5" (UniqueName: "kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5") pod "network-check-target-2rn4s" (UID: "54f69821-e786-41ef-a3bf-41d31c11e2e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:51.342002 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.341962 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh992\" (UniqueName: \"kubernetes.io/projected/955a1098-f214-44eb-bcb0-c73ccaba2be5-kube-api-access-gh992\") pod \"multus-additional-cni-plugins-fv7dr\" (UID: \"955a1098-f214-44eb-bcb0-c73ccaba2be5\") " pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.343093 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.343076 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rh88\" (UniqueName: \"kubernetes.io/projected/90093b1d-bd1c-415c-9273-50bc6c7c84dc-kube-api-access-5rh88\") pod \"tuned-q6nd6\" (UID: \"90093b1d-bd1c-415c-9273-50bc6c7c84dc\") " pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.343399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.343373 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9zz\" (UniqueName: \"kubernetes.io/projected/09458f3b-e5f7-4f3d-b621-5396d4176382-kube-api-access-2l9zz\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:51.343489 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.343448 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chcbh\" (UniqueName: \"kubernetes.io/projected/09688289-2a10-441b-bc83-c7453ea6bb61-kube-api-access-chcbh\") pod \"node-ca-rvnvx\" (UID: \"09688289-2a10-441b-bc83-c7453ea6bb61\") " pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.343563 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.343543 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4ljc\" (UniqueName: \"kubernetes.io/projected/aa080866-0ad2-43b4-90b6-68339fdcffc2-kube-api-access-x4ljc\") pod \"ovnkube-node-vcfvm\" (UID: \"aa080866-0ad2-43b4-90b6-68339fdcffc2\") " pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.343780 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.343765 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns6tk\" (UniqueName: \"kubernetes.io/projected/98ec5735-4c53-4696-b8e0-2dfc7ea09cbc-kube-api-access-ns6tk\") pod \"node-resolver-7fdpb\" (UID: \"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc\") " pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.344275 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.344259 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5vzc\" (UniqueName: \"kubernetes.io/projected/f12f1a7e-9909-4508-b69c-095ffc809462-kube-api-access-d5vzc\") pod \"multus-drpr8\" (UID: \"f12f1a7e-9909-4508-b69c-095ffc809462\") " pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.386681 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.386627 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" event={"ID":"7d9eb1da208d4c1c520252bceb70d657","Type":"ContainerStarted","Data":"fd92e5286927dfa015fae4b525e523d2bace16916d7cb7a6b1549360f033143b"} Jul 06 14:00:51.387565 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.387539 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" event={"ID":"1524ac3c95d4a1a172ad30214b4048df","Type":"ContainerStarted","Data":"57864963e3b0808ee6ca39f7a60e05c74e1703e46d551b01a237cd83e1955725"} Jul 06 14:00:51.434711 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f31c9785-76aa-4397-8383-3d5d06adc6b8-kubelet-config\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/683282e2-842c-44fc-be74-df395fa5876b-host-slash\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434801 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stf8r\" (UniqueName: \"kubernetes.io/projected/683282e2-842c-44fc-be74-df395fa5876b-kube-api-access-stf8r\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-socket-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434821 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-device-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434850 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-registration-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.434882 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/683282e2-842c-44fc-be74-df395fa5876b-iptables-alerter-script\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434874 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/683282e2-842c-44fc-be74-df395fa5876b-host-slash\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434821 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f31c9785-76aa-4397-8383-3d5d06adc6b8-kubelet-config\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434931 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-device-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.434934 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434963 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-registration-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.434973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-sys-fs\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.435007 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret podName:f31c9785-76aa-4397-8383-3d5d06adc6b8 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:51.934988272 +0000 UTC m=+3.124037302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret") pod "global-pull-secret-syncer-9g6dg" (UID: "f31c9785-76aa-4397-8383-3d5d06adc6b8") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.435020 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-sys-fs\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.435020 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.435024 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8485d94b-b2ce-4c28-94eb-0e434864be9f-socket-dir\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.435043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f31c9785-76aa-4397-8383-3d5d06adc6b8-dbus\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.435094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24b7k\" (UniqueName: \"kubernetes.io/projected/8485d94b-b2ce-4c28-94eb-0e434864be9f-kube-api-access-24b7k\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.435373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.435178 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f31c9785-76aa-4397-8383-3d5d06adc6b8-dbus\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.435758 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.435384 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/683282e2-842c-44fc-be74-df395fa5876b-iptables-alerter-script\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.446100 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.446076 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:00:51.449391 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.449339 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stf8r\" (UniqueName: \"kubernetes.io/projected/683282e2-842c-44fc-be74-df395fa5876b-kube-api-access-stf8r\") pod \"iptables-alerter-tdqwr\" (UID: \"683282e2-842c-44fc-be74-df395fa5876b\") " pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.449860 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.449844 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24b7k\" (UniqueName: \"kubernetes.io/projected/8485d94b-b2ce-4c28-94eb-0e434864be9f-kube-api-access-24b7k\") pod \"aws-ebs-csi-driver-node-8t69v\" (UID: \"8485d94b-b2ce-4c28-94eb-0e434864be9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.518892 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.518859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:00:51.526663 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.526640 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" Jul 06 14:00:51.527089 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.527061 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32a7a04_eee4_4d9d_8026_97d8501d6d50.slice/crio-64ef414d933fa579392729217a2f6f19e4ca706ff9c77457269ba06d5e50355f WatchSource:0}: Error finding container 64ef414d933fa579392729217a2f6f19e4ca706ff9c77457269ba06d5e50355f: Status 404 returned error can't find the container with id 64ef414d933fa579392729217a2f6f19e4ca706ff9c77457269ba06d5e50355f Jul 06 14:00:51.534394 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.534373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7fdpb" Jul 06 14:00:51.534497 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.534466 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90093b1d_bd1c_415c_9273_50bc6c7c84dc.slice/crio-841e3f49535e23f9f659dbd8a2bc02cbaa0137d517813a0fa80fce16dc90dee3 WatchSource:0}: Error finding container 841e3f49535e23f9f659dbd8a2bc02cbaa0137d517813a0fa80fce16dc90dee3: Status 404 returned error can't find the container with id 841e3f49535e23f9f659dbd8a2bc02cbaa0137d517813a0fa80fce16dc90dee3 Jul 06 14:00:51.538526 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.538509 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-drpr8" Jul 06 14:00:51.542244 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.542222 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ec5735_4c53_4696_b8e0_2dfc7ea09cbc.slice/crio-0e07ae6365dabf06a0643091364b29882d8f8809c55df70c78e182ac1f4e90b9 WatchSource:0}: Error finding container 0e07ae6365dabf06a0643091364b29882d8f8809c55df70c78e182ac1f4e90b9: Status 404 returned error can't find the container with id 0e07ae6365dabf06a0643091364b29882d8f8809c55df70c78e182ac1f4e90b9 Jul 06 14:00:51.543417 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.543399 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" Jul 06 14:00:51.546208 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.546190 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12f1a7e_9909_4508_b69c_095ffc809462.slice/crio-f9e4c59d016a4f2d980c1717246f318b8a08f48991ddc9255e84baf0eb801bf9 WatchSource:0}: Error finding container f9e4c59d016a4f2d980c1717246f318b8a08f48991ddc9255e84baf0eb801bf9: Status 404 returned error can't find the container with id f9e4c59d016a4f2d980c1717246f318b8a08f48991ddc9255e84baf0eb801bf9 Jul 06 14:00:51.550352 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.550332 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:00:51.550525 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.550501 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955a1098_f214_44eb_bcb0_c73ccaba2be5.slice/crio-64b2ec7f9efd034291505ddd56e64ae5497b69410785cf7a168151e9b3aad8ff WatchSource:0}: Error finding container 64b2ec7f9efd034291505ddd56e64ae5497b69410785cf7a168151e9b3aad8ff: Status 404 returned error can't find the container with id 64b2ec7f9efd034291505ddd56e64ae5497b69410785cf7a168151e9b3aad8ff Jul 06 14:00:51.555800 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.555784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rvnvx" Jul 06 14:00:51.558407 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.558387 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa080866_0ad2_43b4_90b6_68339fdcffc2.slice/crio-8b6cf2920b32f0e956713fdf8084813e778b7408a040baa2fdce2aa8c9c4e632 WatchSource:0}: Error finding container 8b6cf2920b32f0e956713fdf8084813e778b7408a040baa2fdce2aa8c9c4e632: Status 404 returned error can't find the container with id 8b6cf2920b32f0e956713fdf8084813e778b7408a040baa2fdce2aa8c9c4e632 Jul 06 14:00:51.561405 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.561364 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tdqwr" Jul 06 14:00:51.563211 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.563187 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09688289_2a10_441b_bc83_c7453ea6bb61.slice/crio-96e6ad2b08c62357b5fb547ba20e995c3e6a6b49a7ccd30eaef203ba6985d8eb WatchSource:0}: Error finding container 96e6ad2b08c62357b5fb547ba20e995c3e6a6b49a7ccd30eaef203ba6985d8eb: Status 404 returned error can't find the container with id 96e6ad2b08c62357b5fb547ba20e995c3e6a6b49a7ccd30eaef203ba6985d8eb Jul 06 14:00:51.567124 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.567030 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" Jul 06 14:00:51.570222 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.570203 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683282e2_842c_44fc_be74_df395fa5876b.slice/crio-cce3629fd593b408b982f1d6b9edec490cf4155d3172edab57782594ab2278be WatchSource:0}: Error finding container cce3629fd593b408b982f1d6b9edec490cf4155d3172edab57782594ab2278be: Status 404 returned error can't find the container with id cce3629fd593b408b982f1d6b9edec490cf4155d3172edab57782594ab2278be Jul 06 14:00:51.575239 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:00:51.575218 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8485d94b_b2ce_4c28_94eb_0e434864be9f.slice/crio-64b24019c7a09640987e885d8dfd9ddd4abb1b2130f6cb9444ef9fa9079e9934 WatchSource:0}: Error finding container 64b24019c7a09640987e885d8dfd9ddd4abb1b2130f6cb9444ef9fa9079e9934: Status 404 returned error can't find the container with id 64b24019c7a09640987e885d8dfd9ddd4abb1b2130f6cb9444ef9fa9079e9934 Jul 06 14:00:51.837832 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.837744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:51.838020 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.837956 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:51.838095 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.838054 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs podName:09458f3b-e5f7-4f3d-b621-5396d4176382 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:52.83803079 +0000 UTC m=+4.027079818 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs") pod "network-metrics-daemon-ll8js" (UID: "09458f3b-e5f7-4f3d-b621-5396d4176382") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:51.938857 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.938814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:51.939055 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:51.938879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:51.939055 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.939038 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:51.939167 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.939115 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret podName:f31c9785-76aa-4397-8383-3d5d06adc6b8 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:52.939095201 +0000 UTC m=+4.128144226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret") pod "global-pull-secret-syncer-9g6dg" (UID: "f31c9785-76aa-4397-8383-3d5d06adc6b8") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:51.939554 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.939539 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:00:51.939615 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.939561 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:00:51.939615 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.939575 2562 projected.go:196] Error preparing data for projected volume kube-api-access-djdj5 for pod openshift-network-diagnostics/network-check-target-2rn4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:51.939711 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:51.939620 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5 podName:54f69821-e786-41ef-a3bf-41d31c11e2e2 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:52.9396062 +0000 UTC m=+4.128655231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-djdj5" (UniqueName: "kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5") pod "network-check-target-2rn4s" (UID: "54f69821-e786-41ef-a3bf-41d31c11e2e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:52.277163 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.277088 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 13:55:51 +0000 UTC" deadline="2027-12-15 06:10:50.362924799 +0000 UTC" Jul 06 14:00:52.277163 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.277162 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12640h9m58.085768122s" Jul 06 14:00:52.383384 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.382783 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:52.383384 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.382935 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:00:52.400552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.400471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" event={"ID":"8485d94b-b2ce-4c28-94eb-0e434864be9f","Type":"ContainerStarted","Data":"64b24019c7a09640987e885d8dfd9ddd4abb1b2130f6cb9444ef9fa9079e9934"} Jul 06 14:00:52.409396 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.409236 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"8b6cf2920b32f0e956713fdf8084813e778b7408a040baa2fdce2aa8c9c4e632"} Jul 06 14:00:52.419923 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.419781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7fdpb" event={"ID":"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc","Type":"ContainerStarted","Data":"0e07ae6365dabf06a0643091364b29882d8f8809c55df70c78e182ac1f4e90b9"} Jul 06 14:00:52.430239 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.430203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" event={"ID":"90093b1d-bd1c-415c-9273-50bc6c7c84dc","Type":"ContainerStarted","Data":"841e3f49535e23f9f659dbd8a2bc02cbaa0137d517813a0fa80fce16dc90dee3"} Jul 06 14:00:52.439195 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.439157 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffc78" event={"ID":"b32a7a04-eee4-4d9d-8026-97d8501d6d50","Type":"ContainerStarted","Data":"64ef414d933fa579392729217a2f6f19e4ca706ff9c77457269ba06d5e50355f"} Jul 06 14:00:52.443436 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.443389 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tdqwr" event={"ID":"683282e2-842c-44fc-be74-df395fa5876b","Type":"ContainerStarted","Data":"cce3629fd593b408b982f1d6b9edec490cf4155d3172edab57782594ab2278be"} Jul 06 14:00:52.446179 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.446149 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rvnvx" event={"ID":"09688289-2a10-441b-bc83-c7453ea6bb61","Type":"ContainerStarted","Data":"96e6ad2b08c62357b5fb547ba20e995c3e6a6b49a7ccd30eaef203ba6985d8eb"} Jul 06 14:00:52.453518 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.453487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerStarted","Data":"64b2ec7f9efd034291505ddd56e64ae5497b69410785cf7a168151e9b3aad8ff"} Jul 06 14:00:52.464617 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.464559 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-drpr8" event={"ID":"f12f1a7e-9909-4508-b69c-095ffc809462","Type":"ContainerStarted","Data":"f9e4c59d016a4f2d980c1717246f318b8a08f48991ddc9255e84baf0eb801bf9"} Jul 06 14:00:52.845301 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.845205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:52.845474 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.845384 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:52.845474 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.845444 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs podName:09458f3b-e5f7-4f3d-b621-5396d4176382 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:54.845426144 +0000 UTC m=+6.034475170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs") pod "network-metrics-daemon-ll8js" (UID: "09458f3b-e5f7-4f3d-b621-5396d4176382") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:52.945765 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.945726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:52.945965 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:52.945777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:52.946028 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.945963 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:52.946028 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.946026 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret podName:f31c9785-76aa-4397-8383-3d5d06adc6b8 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:54.946006849 +0000 UTC m=+6.135055880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret") pod "global-pull-secret-syncer-9g6dg" (UID: "f31c9785-76aa-4397-8383-3d5d06adc6b8") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:52.946442 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.946423 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:00:52.946525 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.946451 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:00:52.946525 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.946466 2562 projected.go:196] Error preparing data for projected volume kube-api-access-djdj5 for pod openshift-network-diagnostics/network-check-target-2rn4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:52.946525 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:52.946510 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5 podName:54f69821-e786-41ef-a3bf-41d31c11e2e2 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:54.946495079 +0000 UTC m=+6.135544105 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-djdj5" (UniqueName: "kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5") pod "network-check-target-2rn4s" (UID: "54f69821-e786-41ef-a3bf-41d31c11e2e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:53.278305 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:53.278183 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 13:55:51 +0000 UTC" deadline="2028-01-25 12:03:48.862687524 +0000 UTC" Jul 06 14:00:53.278305 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:53.278221 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13630h2m55.584471137s" Jul 06 14:00:53.385778 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:53.385745 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:53.385965 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:53.385878 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:00:53.386302 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:53.386279 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:53.386416 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:53.386397 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:00:54.382557 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:54.382523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:54.383050 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.382647 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:00:54.863012 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:54.862934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:54.863176 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.863008 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:54.863176 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.863085 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs podName:09458f3b-e5f7-4f3d-b621-5396d4176382 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:58.86306884 +0000 UTC m=+10.052117873 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs") pod "network-metrics-daemon-ll8js" (UID: "09458f3b-e5f7-4f3d-b621-5396d4176382") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:54.963672 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:54.963633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:54.963865 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:54.963721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:54.963865 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.963840 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:00:54.964017 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.963868 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:00:54.964017 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.963883 2562 projected.go:196] Error preparing data for projected volume kube-api-access-djdj5 for pod openshift-network-diagnostics/network-check-target-2rn4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:54.964017 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.963848 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:54.964017 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.963971 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5 podName:54f69821-e786-41ef-a3bf-41d31c11e2e2 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:58.963948618 +0000 UTC m=+10.152997688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-djdj5" (UniqueName: "kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5") pod "network-check-target-2rn4s" (UID: "54f69821-e786-41ef-a3bf-41d31c11e2e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:54.964017 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:54.963991 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret podName:f31c9785-76aa-4397-8383-3d5d06adc6b8 nodeName:}" failed. No retries permitted until 2026-07-06 14:00:58.963981905 +0000 UTC m=+10.153030935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret") pod "global-pull-secret-syncer-9g6dg" (UID: "f31c9785-76aa-4397-8383-3d5d06adc6b8") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:55.385057 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:55.385017 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:55.385057 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:55.385042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:55.385576 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:55.385182 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:00:55.385652 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:55.385604 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:00:56.383890 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:56.383377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:56.383890 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:56.383515 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:00:57.384924 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:57.383409 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:57.384924 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:57.383538 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:00:57.384924 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:57.383980 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:57.384924 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:57.384084 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:00:58.382741 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:58.382698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:58.382925 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.382829 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:00:58.893434 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:58.893389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:58.893890 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.893561 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:58.893890 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.893655 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs podName:09458f3b-e5f7-4f3d-b621-5396d4176382 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:06.893633978 +0000 UTC m=+18.082683008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs") pod "network-metrics-daemon-ll8js" (UID: "09458f3b-e5f7-4f3d-b621-5396d4176382") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:00:58.994341 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:58.994301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:58.994536 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:58.994416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:00:58.994536 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.994484 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:58.994536 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.994532 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:00:58.994684 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.994549 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:00:58.994684 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.994561 2562 projected.go:196] Error preparing data for projected volume kube-api-access-djdj5 for pod openshift-network-diagnostics/network-check-target-2rn4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:58.994684 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.994562 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret podName:f31c9785-76aa-4397-8383-3d5d06adc6b8 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:06.99453958 +0000 UTC m=+18.183588609 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret") pod "global-pull-secret-syncer-9g6dg" (UID: "f31c9785-76aa-4397-8383-3d5d06adc6b8") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:00:58.994684 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:58.994621 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5 podName:54f69821-e786-41ef-a3bf-41d31c11e2e2 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:06.994604571 +0000 UTC m=+18.183653600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-djdj5" (UniqueName: "kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5") pod "network-check-target-2rn4s" (UID: "54f69821-e786-41ef-a3bf-41d31c11e2e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:00:59.384150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:59.383569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:00:59.384150 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:59.383935 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:00:59.384150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:00:59.383744 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:00:59.384150 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:00:59.384053 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:00.382674 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:00.382623 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:00.383118 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:00.382793 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:01.383884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:01.383701 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:01.383884 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:01.383837 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:01.384377 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:01.383714 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:01.384377 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:01.384318 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:02.382692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:02.382655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:02.382882 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:02.382809 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:03.383230 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:03.383143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:03.383682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:03.383143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:03.383682 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:03.383299 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:03.383682 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:03.383361 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:04.382795 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:04.382760 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:04.383000 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:04.382878 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:05.382873 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:05.382832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:05.382873 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:05.382874 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:05.383424 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:05.383016 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:05.383424 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:05.383148 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:06.382824 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:06.382788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:06.383067 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:06.382930 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:06.956217 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:06.956169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:06.956402 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:06.956321 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:01:06.956402 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:06.956392 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs podName:09458f3b-e5f7-4f3d-b621-5396d4176382 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:22.956374538 +0000 UTC m=+34.145423565 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs") pod "network-metrics-daemon-ll8js" (UID: "09458f3b-e5f7-4f3d-b621-5396d4176382") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:01:07.057407 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:07.057360 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:07.057407 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:07.057415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:07.057660 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.057516 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:01:07.057660 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.057537 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:01:07.057660 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.057565 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:01:07.057660 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.057580 2562 projected.go:196] Error preparing data for projected volume kube-api-access-djdj5 for pod openshift-network-diagnostics/network-check-target-2rn4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:01:07.057660 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.057584 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret podName:f31c9785-76aa-4397-8383-3d5d06adc6b8 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:23.057564416 +0000 UTC m=+34.246613465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret") pod "global-pull-secret-syncer-9g6dg" (UID: "f31c9785-76aa-4397-8383-3d5d06adc6b8") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:01:07.057660 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.057625 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5 podName:54f69821-e786-41ef-a3bf-41d31c11e2e2 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:23.057613718 +0000 UTC m=+34.246662745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-djdj5" (UniqueName: "kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5") pod "network-check-target-2rn4s" (UID: "54f69821-e786-41ef-a3bf-41d31c11e2e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:01:07.383149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:07.383110 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:07.383606 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.383262 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:07.383606 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:07.383304 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:07.383606 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:07.383398 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:08.383348 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:08.383312 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:08.383823 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:08.383429 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:09.383406 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:09.383368 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:09.383878 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:09.383440 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:09.383878 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:09.383464 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:09.383878 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:09.383539 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:10.383092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:10.383060 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:10.383262 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:10.383167 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:11.383160 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.383129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:11.383160 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.383159 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:11.383543 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:11.383249 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:11.383543 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:11.383326 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:11.518194 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.517893 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" event={"ID":"1524ac3c95d4a1a172ad30214b4048df","Type":"ContainerStarted","Data":"049986e9bab77064967c2bb2cc0a395052e8e16cbb50ec6229c94095797ea175"} Jul 06 14:01:11.529547 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.529423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"36103f691aa634cffa172c260d5db427087ec222f79c75e46582090a086f8f91"} Jul 06 14:01:11.529547 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.529514 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"70a388428351ed2048d978c17294492dddea484b953d0a4281b637ecdb291dfa"} Jul 06 14:01:11.531254 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.531219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" event={"ID":"90093b1d-bd1c-415c-9273-50bc6c7c84dc","Type":"ContainerStarted","Data":"58717dd2ca3bef47c6dc577e86ab80c73a66dd873a82203fb778e0432afdba99"} Jul 06 14:01:11.536355 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.536301 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-92.ec2.internal" podStartSLOduration=21.536282319 podStartE2EDuration="21.536282319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:11.535841856 +0000 UTC m=+22.724890902" watchObservedRunningTime="2026-07-06 14:01:11.536282319 +0000 UTC m=+22.725331367" Jul 06 14:01:11.563159 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:11.563088 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q6nd6" podStartSLOduration=2.99078365 podStartE2EDuration="22.563065057s" totalImagesPullingTime="19.572281407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:11.56186657 +0000 UTC m=+22.750915618" watchObservedRunningTime="2026-07-06 14:01:11.563065057 +0000 UTC m=+22.752114104" Jul 06 14:01:12.383152 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.383122 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:12.383817 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:12.383224 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:12.536539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.536466 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:01:12.536803 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.536784 2562 generic.go:356] "Generic (PLEG): container finished" podID="aa080866-0ad2-43b4-90b6-68339fdcffc2" containerID="36103f691aa634cffa172c260d5db427087ec222f79c75e46582090a086f8f91" exitCode=1 Jul 06 14:01:12.536883 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.536854 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerDied","Data":"36103f691aa634cffa172c260d5db427087ec222f79c75e46582090a086f8f91"} Jul 06 14:01:12.536983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.536966 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"f353eb3751765fc691b6ed897ee4a0926060e994618efac2ecdda609f580d2b0"} Jul 06 14:01:12.537038 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.537022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"4f647fb923b775670c2cb93083d06c0d0079f230a348685c8d281a6e198f5785"} Jul 06 14:01:12.537083 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.537070 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"d165afd64e8b06d984cd6a6a0913f7fb57caaacf3277dba099e6679ca7e990ae"} Jul 06 14:01:12.537132 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.537118 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"735fcd951de16a9d0324a043ff7d71cacc6645e069bfaea0d48319cad514bb39"} Jul 06 14:01:12.538310 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.538287 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7fdpb" event={"ID":"98ec5735-4c53-4696-b8e0-2dfc7ea09cbc","Type":"ContainerStarted","Data":"45553022095f6b3a4f19c91cf342dafbae989023a9de9ea93aa806f5dc4d9675"} Jul 06 14:01:12.539674 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.539648 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffc78" event={"ID":"b32a7a04-eee4-4d9d-8026-97d8501d6d50","Type":"ContainerStarted","Data":"9ff3a863d17aacc5b3ae8d07114c45c53abc08e6ccce343832b51b5884fa426e"} Jul 06 14:01:12.541062 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.541040 2562 generic.go:356] "Generic (PLEG): container finished" podID="7d9eb1da208d4c1c520252bceb70d657" containerID="46b2ce5a325811f4c98cd4625a35404521f601f4d140a54c6ab0066e8dc3b220" exitCode=0 Jul 06 14:01:12.541158 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.541115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" event={"ID":"7d9eb1da208d4c1c520252bceb70d657","Type":"ContainerDied","Data":"46b2ce5a325811f4c98cd4625a35404521f601f4d140a54c6ab0066e8dc3b220"} Jul 06 14:01:12.542707 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.542684 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rvnvx" event={"ID":"09688289-2a10-441b-bc83-c7453ea6bb61","Type":"ContainerStarted","Data":"07a6dc19f744278d97c1604e1bb20671473f6818422881d7a617811811917c2b"} Jul 06 14:01:12.544231 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.544210 2562 generic.go:356] "Generic (PLEG): container finished" podID="955a1098-f214-44eb-bcb0-c73ccaba2be5" containerID="ba2e03162c43e9cdb56319962668262551ad83483ad3f8f2638a256e0f8163fb" exitCode=0 Jul 06 14:01:12.544326 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.544274 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerDied","Data":"ba2e03162c43e9cdb56319962668262551ad83483ad3f8f2638a256e0f8163fb"} Jul 06 14:01:12.545742 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.545719 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-drpr8" event={"ID":"f12f1a7e-9909-4508-b69c-095ffc809462","Type":"ContainerStarted","Data":"0e4bc58e6003a40a84f90bf999fca8f5c21794eb2f8d3c6462e23663108cb062"} Jul 06 14:01:12.547137 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.547120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" event={"ID":"8485d94b-b2ce-4c28-94eb-0e434864be9f","Type":"ContainerStarted","Data":"14049126ac054d7069f7fa3a3f8f30a9ee54781db916a216e860cf00dc1d7df8"} Jul 06 14:01:12.555485 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.555448 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7fdpb" podStartSLOduration=3.863621143 podStartE2EDuration="23.555436125s" totalImagesPullingTime="19.691814982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:12.555359749 +0000 UTC m=+23.744408807" watchObservedRunningTime="2026-07-06 14:01:12.555436125 +0000 UTC m=+23.744485171" Jul 06 14:01:12.575521 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.575481 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-drpr8" podStartSLOduration=3.689713194 podStartE2EDuration="23.575466645s" totalImagesPullingTime="19.885753451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:12.575183015 +0000 UTC m=+23.764232062" watchObservedRunningTime="2026-07-06 14:01:12.575466645 +0000 UTC m=+23.764515691" Jul 06 14:01:12.635409 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.635364 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ffc78" podStartSLOduration=7.901718286 podStartE2EDuration="23.63534868s" totalImagesPullingTime="15.733630394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:12.63518311 +0000 UTC m=+23.824232156" watchObservedRunningTime="2026-07-06 14:01:12.63534868 +0000 UTC m=+23.824397726" Jul 06 14:01:12.654674 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:12.654633 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rvnvx" podStartSLOduration=7.959150578 podStartE2EDuration="23.654617837s" totalImagesPullingTime="15.695467259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:12.654184934 +0000 UTC m=+23.843233981" watchObservedRunningTime="2026-07-06 14:01:12.654617837 +0000 UTC m=+23.843666884" Jul 06 14:01:13.382694 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:13.382656 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:13.382917 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:13.382656 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:13.382917 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:13.382867 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:13.382917 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:13.382774 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:13.551754 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:13.551707 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" event={"ID":"7d9eb1da208d4c1c520252bceb70d657","Type":"ContainerStarted","Data":"b81d9fdfc35539b3540a9b9883aa800df7ccd9d9db78b69c41df7e77afec8ef9"} Jul 06 14:01:13.553622 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:13.553587 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tdqwr" event={"ID":"683282e2-842c-44fc-be74-df395fa5876b","Type":"ContainerStarted","Data":"c80499617b527ca74b87730bf5b5595fff85400ab4dc8ac6479e44d08fcdb4e0"} Jul 06 14:01:13.569537 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:13.569480 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-92.ec2.internal" podStartSLOduration=23.56946417 podStartE2EDuration="23.56946417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:13.568669544 +0000 UTC m=+24.757718592" watchObservedRunningTime="2026-07-06 14:01:13.56946417 +0000 UTC m=+24.758513222" Jul 06 14:01:13.587505 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:13.587453 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tdqwr" podStartSLOduration=4.922721995 podStartE2EDuration="24.587438942s" totalImagesPullingTime="19.664716947s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:13.586809987 +0000 UTC m=+24.775859038" watchObservedRunningTime="2026-07-06 14:01:13.587438942 +0000 UTC m=+24.776488009" Jul 06 14:01:13.833486 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:13.833464 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 14:01:14.290157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:14.290023 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T14:01:13.833480715Z","UUID":"9603146b-8a96-4394-aa33-c05559c2f440","Handler":null,"Name":"","Endpoint":""} Jul 06 14:01:14.291442 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:14.291416 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 14:01:14.291442 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:14.291448 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 14:01:14.383235 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:14.383189 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:14.383416 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:14.383338 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:14.557362 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:14.557274 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" event={"ID":"8485d94b-b2ce-4c28-94eb-0e434864be9f","Type":"ContainerStarted","Data":"32c4ce24c72c39710cefa28b7496efe7ff3fa05069aa31f06d7f26270c7ffb91"} Jul 06 14:01:14.560195 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:14.560175 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:01:14.560568 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:14.560539 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"5ac0ffad88232b90176a77ce343cd47385eea4d5be7653e06e8fc6e91a7c3a4f"} Jul 06 14:01:15.383192 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:15.383105 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:15.383346 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:15.383124 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:15.383346 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:15.383236 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:15.383346 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:15.383301 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:15.565088 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:15.565047 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" event={"ID":"8485d94b-b2ce-4c28-94eb-0e434864be9f","Type":"ContainerStarted","Data":"6f7a63791d94dabd6be8473ca38c598fcdcf0d5e657cbd62d45ba9b9a6997499"} Jul 06 14:01:15.590872 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:15.590823 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8t69v" podStartSLOduration=4.473371847 podStartE2EDuration="26.590807053s" totalImagesPullingTime="22.117435206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:15.58908464 +0000 UTC m=+26.778133706" watchObservedRunningTime="2026-07-06 14:01:15.590807053 +0000 UTC m=+26.779856099" Jul 06 14:01:16.354169 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.353937 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:01:16.354545 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.354526 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:01:16.382851 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.382819 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:16.383021 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:16.382920 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:16.570625 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.570602 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:01:16.571321 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.570960 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"a82c10396c15c6df05824ae0565e785013fe18d27bec996323fd95baeca4afd0"} Jul 06 14:01:16.571479 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.571431 2562 scope.go:117] "RemoveContainer" containerID="36103f691aa634cffa172c260d5db427087ec222f79c75e46582090a086f8f91" Jul 06 14:01:16.571623 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.571602 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:01:16.571724 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.571652 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:01:16.571724 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.571702 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:01:16.571824 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.571749 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:01:16.571873 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.571824 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ffc78" Jul 06 14:01:16.587271 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.587241 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:01:16.587687 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.587666 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:01:16.673083 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:16.673060 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7fdpb_98ec5735-4c53-4696-b8e0-2dfc7ea09cbc/dns-node-resolver/0.log" Jul 06 14:01:17.382964 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:17.382929 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:17.382964 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:17.382947 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:17.383201 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:17.383058 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:17.383268 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:17.383195 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:17.577443 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:17.577412 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:01:17.578085 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:17.577799 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" event={"ID":"aa080866-0ad2-43b4-90b6-68339fdcffc2","Type":"ContainerStarted","Data":"9e43f03e9215992a333173e9a48f328c1dc5731ae7bd93d6769bff0e434c8616"} Jul 06 14:01:17.628004 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:17.627946 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" podStartSLOduration=8.949115829 podStartE2EDuration="28.62792906s" totalImagesPullingTime="19.678813231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:17.627242859 +0000 UTC m=+28.816291924" watchObservedRunningTime="2026-07-06 14:01:17.62792906 +0000 UTC m=+28.816978107" Jul 06 14:01:17.835141 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:17.835053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rvnvx_09688289-2a10-441b-bc83-c7453ea6bb61/node-ca/0.log" Jul 06 14:01:18.245596 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:18.245267 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ll8js"] Jul 06 14:01:18.245746 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:18.245643 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:18.245795 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:18.245745 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:18.248050 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:18.248025 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2rn4s"] Jul 06 14:01:18.248184 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:18.248113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:18.248250 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:18.248208 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:18.255143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:18.255117 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9g6dg"] Jul 06 14:01:18.255274 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:18.255226 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:18.255350 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:18.255328 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:19.384086 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:19.384000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:19.384438 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:19.384103 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:19.584764 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:19.584734 2562 generic.go:356] "Generic (PLEG): container finished" podID="955a1098-f214-44eb-bcb0-c73ccaba2be5" containerID="ade174e9bf52d86b574ffeff26a67851843033d657b14f34524c6b05ce835045" exitCode=0 Jul 06 14:01:19.584933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:19.584789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerDied","Data":"ade174e9bf52d86b574ffeff26a67851843033d657b14f34524c6b05ce835045"} Jul 06 14:01:20.382832 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:20.382795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:20.382832 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:20.382821 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:20.383159 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:20.382942 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:20.383159 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:20.383046 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:21.382783 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:21.382752 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:21.383159 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:21.382876 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:21.591710 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:21.591680 2562 generic.go:356] "Generic (PLEG): container finished" podID="955a1098-f214-44eb-bcb0-c73ccaba2be5" containerID="f8f0fb5c8390faf85293868020ef84246a0221433a1048e604415221d2c401d8" exitCode=0 Jul 06 14:01:21.591849 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:21.591734 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerDied","Data":"f8f0fb5c8390faf85293868020ef84246a0221433a1048e604415221d2c401d8"} Jul 06 14:01:22.382579 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:22.382547 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:22.382717 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:22.382575 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:22.382717 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:22.382692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:22.382807 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:22.382790 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:22.596011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:22.595755 2562 generic.go:356] "Generic (PLEG): container finished" podID="955a1098-f214-44eb-bcb0-c73ccaba2be5" containerID="0c62c3c46a5d2b297505bbf23a1110d0dad8fc1f790551ecf4ce06df013a8dd0" exitCode=0 Jul 06 14:01:22.596011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:22.595843 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerDied","Data":"0c62c3c46a5d2b297505bbf23a1110d0dad8fc1f790551ecf4ce06df013a8dd0"} Jul 06 14:01:22.987842 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:22.987738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:22.988035 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:22.987876 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:01:22.988035 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:22.987964 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs podName:09458f3b-e5f7-4f3d-b621-5396d4176382 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:54.987945217 +0000 UTC m=+66.176994248 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs") pod "network-metrics-daemon-ll8js" (UID: "09458f3b-e5f7-4f3d-b621-5396d4176382") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:01:23.088294 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:23.088227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:23.088294 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:23.088271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:23.088556 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:23.088379 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:01:23.088556 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:23.088390 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:01:23.088556 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:23.088411 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:01:23.088556 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:23.088422 2562 projected.go:196] Error preparing data for projected volume kube-api-access-djdj5 for pod openshift-network-diagnostics/network-check-target-2rn4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:01:23.088556 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:23.088442 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret podName:f31c9785-76aa-4397-8383-3d5d06adc6b8 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:55.088424854 +0000 UTC m=+66.277473879 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret") pod "global-pull-secret-syncer-9g6dg" (UID: "f31c9785-76aa-4397-8383-3d5d06adc6b8") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:01:23.088556 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:23.088461 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5 podName:54f69821-e786-41ef-a3bf-41d31c11e2e2 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:55.088448931 +0000 UTC m=+66.277497959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-djdj5" (UniqueName: "kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5") pod "network-check-target-2rn4s" (UID: "54f69821-e786-41ef-a3bf-41d31c11e2e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:01:23.382645 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:23.382613 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:23.382808 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:23.382719 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:24.383122 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:24.383087 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:24.383728 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:24.383092 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:24.383728 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:24.383222 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:24.383728 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:24.383342 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:25.383459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:25.383426 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:25.384055 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:25.383566 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:26.383185 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:26.383149 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:26.383382 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:26.383147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:26.383382 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:26.383275 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:26.383382 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:26.383327 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:27.383396 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:27.383361 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:27.383869 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:27.383501 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:28.383483 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:28.383459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:28.383874 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:28.383494 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:28.383874 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:28.383566 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:28.383874 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:28.383690 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:28.612080 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:28.611984 2562 generic.go:356] "Generic (PLEG): container finished" podID="955a1098-f214-44eb-bcb0-c73ccaba2be5" containerID="25062f16b44119e6f95843369106cf87c86fdf3d156a1b026c5283ace86ba8d4" exitCode=0 Jul 06 14:01:28.612247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:28.612076 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerDied","Data":"25062f16b44119e6f95843369106cf87c86fdf3d156a1b026c5283ace86ba8d4"} Jul 06 14:01:29.383943 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:29.383884 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:29.384467 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:29.384020 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:29.617117 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:29.617087 2562 generic.go:356] "Generic (PLEG): container finished" podID="955a1098-f214-44eb-bcb0-c73ccaba2be5" containerID="7a9652f99f44fb280ffc90d92aff16f69f63f5762894b19b5aae7880765a2519" exitCode=0 Jul 06 14:01:29.617270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:29.617139 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerDied","Data":"7a9652f99f44fb280ffc90d92aff16f69f63f5762894b19b5aae7880765a2519"} Jul 06 14:01:30.383081 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:30.383045 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:30.383257 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:30.383089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:30.383257 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:30.383156 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:30.383335 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:30.383257 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:30.622498 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:30.622463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" event={"ID":"955a1098-f214-44eb-bcb0-c73ccaba2be5","Type":"ContainerStarted","Data":"c7e76943718f59cfd858e5569306937fa9414430e668a25e632092b0c8b6ea9d"} Jul 06 14:01:30.649463 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:30.649370 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fv7dr" podStartSLOduration=8.003907082 podStartE2EDuration="41.649354001s" totalImagesPullingTime="33.645446919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:30.649169144 +0000 UTC m=+41.838218192" watchObservedRunningTime="2026-07-06 14:01:30.649354001 +0000 UTC m=+41.838403047" Jul 06 14:01:31.383640 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:31.383603 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:31.383798 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:31.383718 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:32.382867 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:32.382826 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:32.382867 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:32.382868 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:32.383290 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:32.382975 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:32.383290 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:32.383082 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:33.382746 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:33.382712 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:33.382930 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:33.382825 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:34.382544 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:34.382510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:34.382712 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:34.382618 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:34.382712 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:34.382667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:34.382784 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:34.382748 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:35.383325 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:35.383295 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:35.383690 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:35.383400 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:36.383042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:36.383004 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:36.383241 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:36.383003 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:36.383241 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:36.383109 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:36.383241 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:36.383200 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:37.383042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:37.383004 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:37.383416 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:37.383176 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ll8js" podUID="09458f3b-e5f7-4f3d-b621-5396d4176382" Jul 06 14:01:38.382757 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:38.382729 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:38.382931 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:38.382836 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9g6dg" podUID="f31c9785-76aa-4397-8383-3d5d06adc6b8" Jul 06 14:01:38.382931 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:38.382887 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:38.383016 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:38.382992 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2rn4s" podUID="54f69821-e786-41ef-a3bf-41d31c11e2e2" Jul 06 14:01:39.134061 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.133974 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-92.ec2.internal" event="NodeReady" Jul 06 14:01:39.134493 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.134125 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 14:01:39.179874 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.179844 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-d7kjp"] Jul 06 14:01:39.197875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.197826 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9d579"] Jul 06 14:01:39.198044 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.197962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.200683 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.200647 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 14:01:39.200683 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.200654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gmrh7\"" Jul 06 14:01:39.200886 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.200700 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 14:01:39.209869 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.209848 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d7kjp"] Jul 06 14:01:39.209980 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.209881 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9d579"] Jul 06 14:01:39.209980 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.209927 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bdv7g"] Jul 06 14:01:39.210064 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.210053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.213723 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.213327 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 14:01:39.213723 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.213358 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 14:01:39.213723 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.213378 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 14:01:39.213723 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.213592 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 14:01:39.213723 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.213697 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fb278\"" Jul 06 14:01:39.219851 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.219826 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdv7g"] Jul 06 14:01:39.219967 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.219931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.223251 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.223233 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xxzpd\"" Jul 06 14:01:39.223533 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.223511 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 14:01:39.223642 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.223617 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 14:01:39.223642 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.223635 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 14:01:39.303234 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303198 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-tmp-dir\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.303234 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/691adf51-ea2f-4aca-aaea-c357787cac97-data-volume\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.303470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dml8n\" (UniqueName: \"kubernetes.io/projected/691adf51-ea2f-4aca-aaea-c357787cac97-kube-api-access-dml8n\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.303470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/691adf51-ea2f-4aca-aaea-c357787cac97-crio-socket\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.303470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-metrics-tls\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.303470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/691adf51-ea2f-4aca-aaea-c357787cac97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.303470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/691adf51-ea2f-4aca-aaea-c357787cac97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.303631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-config-volume\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.303631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.303499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs99j\" (UniqueName: \"kubernetes.io/projected/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-kube-api-access-qs99j\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.384119 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.384036 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:39.387033 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.387009 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hqzzg\"" Jul 06 14:01:39.387033 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.387009 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:01:39.404067 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/691adf51-ea2f-4aca-aaea-c357787cac97-crio-socket\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404195 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404136 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-metrics-tls\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.404195 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddafc31d-a61f-4e4d-87c9-d70e43da0637-cert\") pod \"ingress-canary-bdv7g\" (UID: \"ddafc31d-a61f-4e4d-87c9-d70e43da0637\") " pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.404332 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404202 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/691adf51-ea2f-4aca-aaea-c357787cac97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404332 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404292 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/691adf51-ea2f-4aca-aaea-c357787cac97-crio-socket\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404332 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/691adf51-ea2f-4aca-aaea-c357787cac97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404482 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-config-volume\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.404482 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs99j\" (UniqueName: \"kubernetes.io/projected/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-kube-api-access-qs99j\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.404482 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntgs4\" (UniqueName: \"kubernetes.io/projected/ddafc31d-a61f-4e4d-87c9-d70e43da0637-kube-api-access-ntgs4\") pod \"ingress-canary-bdv7g\" (UID: \"ddafc31d-a61f-4e4d-87c9-d70e43da0637\") " pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.404482 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-tmp-dir\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.404656 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/691adf51-ea2f-4aca-aaea-c357787cac97-data-volume\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404778 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dml8n\" (UniqueName: \"kubernetes.io/projected/691adf51-ea2f-4aca-aaea-c357787cac97-kube-api-access-dml8n\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404841 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404795 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/691adf51-ea2f-4aca-aaea-c357787cac97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404841 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404795 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-tmp-dir\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.404841 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404828 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/691adf51-ea2f-4aca-aaea-c357787cac97-data-volume\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.404995 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.404969 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-config-volume\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.409368 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.409348 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/691adf51-ea2f-4aca-aaea-c357787cac97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.409469 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.409386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-metrics-tls\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.413072 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.413046 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs99j\" (UniqueName: \"kubernetes.io/projected/d2a56be5-b85c-4abb-828b-ea42f00a8f7d-kube-api-access-qs99j\") pod \"dns-default-d7kjp\" (UID: \"d2a56be5-b85c-4abb-828b-ea42f00a8f7d\") " pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.413179 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.413089 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dml8n\" (UniqueName: \"kubernetes.io/projected/691adf51-ea2f-4aca-aaea-c357787cac97-kube-api-access-dml8n\") pod \"insights-runtime-extractor-9d579\" (UID: \"691adf51-ea2f-4aca-aaea-c357787cac97\") " pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.505195 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.505161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddafc31d-a61f-4e4d-87c9-d70e43da0637-cert\") pod \"ingress-canary-bdv7g\" (UID: \"ddafc31d-a61f-4e4d-87c9-d70e43da0637\") " pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.505366 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.505211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntgs4\" (UniqueName: \"kubernetes.io/projected/ddafc31d-a61f-4e4d-87c9-d70e43da0637-kube-api-access-ntgs4\") pod \"ingress-canary-bdv7g\" (UID: \"ddafc31d-a61f-4e4d-87c9-d70e43da0637\") " pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.507263 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.507233 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:39.507547 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.507527 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddafc31d-a61f-4e4d-87c9-d70e43da0637-cert\") pod \"ingress-canary-bdv7g\" (UID: \"ddafc31d-a61f-4e4d-87c9-d70e43da0637\") " pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.513853 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.513827 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntgs4\" (UniqueName: \"kubernetes.io/projected/ddafc31d-a61f-4e4d-87c9-d70e43da0637-kube-api-access-ntgs4\") pod \"ingress-canary-bdv7g\" (UID: \"ddafc31d-a61f-4e4d-87c9-d70e43da0637\") " pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.520787 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.520763 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9d579" Jul 06 14:01:39.528922 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.528887 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bdv7g" Jul 06 14:01:39.719747 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.719703 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9d579"] Jul 06 14:01:39.724292 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:39.724260 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691adf51_ea2f_4aca_aaea_c357787cac97.slice/crio-e403c28a2a47322f05f1c7bf47a0ac0a0fa9f54a52a47be2ec64f289b2d731d0 WatchSource:0}: Error finding container e403c28a2a47322f05f1c7bf47a0ac0a0fa9f54a52a47be2ec64f289b2d731d0: Status 404 returned error can't find the container with id e403c28a2a47322f05f1c7bf47a0ac0a0fa9f54a52a47be2ec64f289b2d731d0 Jul 06 14:01:39.738700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.738669 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d7kjp"] Jul 06 14:01:39.741246 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:39.741218 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a56be5_b85c_4abb_828b_ea42f00a8f7d.slice/crio-b62d54b677d4953835b10a933b190b50e12b0507d61c98565bdc32fddcbd0e2a WatchSource:0}: Error finding container b62d54b677d4953835b10a933b190b50e12b0507d61c98565bdc32fddcbd0e2a: Status 404 returned error can't find the container with id b62d54b677d4953835b10a933b190b50e12b0507d61c98565bdc32fddcbd0e2a Jul 06 14:01:39.750812 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:39.750789 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bdv7g"] Jul 06 14:01:39.753785 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:39.753764 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddafc31d_a61f_4e4d_87c9_d70e43da0637.slice/crio-e7365dfe58d788b182deb95f7b7f7832c252d3774e36e80ecda6c810b99c0d27 WatchSource:0}: Error finding container e7365dfe58d788b182deb95f7b7f7832c252d3774e36e80ecda6c810b99c0d27: Status 404 returned error can't find the container with id e7365dfe58d788b182deb95f7b7f7832c252d3774e36e80ecda6c810b99c0d27 Jul 06 14:01:40.383040 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.383004 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:40.383505 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.383289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:40.386648 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.386403 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q6brz\"" Jul 06 14:01:40.386648 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.386460 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:01:40.388634 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.388146 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:01:40.388634 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.388371 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:01:40.652324 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.652101 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9d579" event={"ID":"691adf51-ea2f-4aca-aaea-c357787cac97","Type":"ContainerStarted","Data":"e403c28a2a47322f05f1c7bf47a0ac0a0fa9f54a52a47be2ec64f289b2d731d0"} Jul 06 14:01:40.654405 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.654273 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdv7g" event={"ID":"ddafc31d-a61f-4e4d-87c9-d70e43da0637","Type":"ContainerStarted","Data":"e7365dfe58d788b182deb95f7b7f7832c252d3774e36e80ecda6c810b99c0d27"} Jul 06 14:01:40.656701 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:40.656596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7kjp" event={"ID":"d2a56be5-b85c-4abb-828b-ea42f00a8f7d","Type":"ContainerStarted","Data":"b62d54b677d4953835b10a933b190b50e12b0507d61c98565bdc32fddcbd0e2a"} Jul 06 14:01:41.127646 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.127612 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp"] Jul 06 14:01:41.137698 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.137653 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" Jul 06 14:01:41.140033 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.140003 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp"] Jul 06 14:01:41.140595 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.140574 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 14:01:41.140739 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.140718 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cc6rh\"" Jul 06 14:01:41.321082 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.321039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/380d419c-c9e9-44fa-abb6-fab97c53fb75-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-gdmkp\" (UID: \"380d419c-c9e9-44fa-abb6-fab97c53fb75\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" Jul 06 14:01:41.345843 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.345807 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56b5c88994-vsnlm"] Jul 06 14:01:41.360935 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.360339 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.363415 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.361273 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b5c88994-vsnlm"] Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365502 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365545 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365691 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365787 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365792 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365863 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sl7tc\"" Jul 06 14:01:41.366001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.365971 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 14:01:41.422618 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.422544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/380d419c-c9e9-44fa-abb6-fab97c53fb75-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-gdmkp\" (UID: \"380d419c-c9e9-44fa-abb6-fab97c53fb75\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" Jul 06 14:01:41.425701 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.425674 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/380d419c-c9e9-44fa-abb6-fab97c53fb75-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-gdmkp\" (UID: \"380d419c-c9e9-44fa-abb6-fab97c53fb75\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" Jul 06 14:01:41.451468 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.451438 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" Jul 06 14:01:41.523586 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.523551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-oauth-serving-cert\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.523727 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.523597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltbcr\" (UniqueName: \"kubernetes.io/projected/348fadc9-b714-4bdc-983a-bfe26b9ebfac-kube-api-access-ltbcr\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.523727 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.523636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-serving-cert\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.523727 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.523694 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-service-ca\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.523914 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.523788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-oauth-config\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.523914 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.523825 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-config\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.618890 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.618815 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp"] Jul 06 14:01:41.625342 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.625307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-oauth-serving-cert\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.625533 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.625357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltbcr\" (UniqueName: \"kubernetes.io/projected/348fadc9-b714-4bdc-983a-bfe26b9ebfac-kube-api-access-ltbcr\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.625533 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.625394 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-serving-cert\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.625533 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.625432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-service-ca\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.625533 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.625485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-oauth-config\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.625533 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.625514 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-config\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.626247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.626224 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-oauth-serving-cert\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.626339 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.626248 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-config\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.626339 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.626269 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-service-ca\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.629094 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.629064 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-serving-cert\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.630555 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.630512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-oauth-config\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.640954 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.640925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltbcr\" (UniqueName: \"kubernetes.io/projected/348fadc9-b714-4bdc-983a-bfe26b9ebfac-kube-api-access-ltbcr\") pod \"console-56b5c88994-vsnlm\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:41.675605 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:41.675529 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:43.493785 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:43.493749 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380d419c_c9e9_44fa_abb6_fab97c53fb75.slice/crio-c013abecb02326b031e46b7da41ab9ba3c85db982a13b739c46236a0156294bc WatchSource:0}: Error finding container c013abecb02326b031e46b7da41ab9ba3c85db982a13b739c46236a0156294bc: Status 404 returned error can't find the container with id c013abecb02326b031e46b7da41ab9ba3c85db982a13b739c46236a0156294bc Jul 06 14:01:43.666714 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:43.666657 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" event={"ID":"380d419c-c9e9-44fa-abb6-fab97c53fb75","Type":"ContainerStarted","Data":"c013abecb02326b031e46b7da41ab9ba3c85db982a13b739c46236a0156294bc"} Jul 06 14:01:45.243664 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.243632 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b5c88994-vsnlm"] Jul 06 14:01:45.378911 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:45.378813 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod348fadc9_b714_4bdc_983a_bfe26b9ebfac.slice/crio-b521ca27473fd48abc0235bbd9e4c48abeeaf34b297e084434b8f51dd25ef2fc WatchSource:0}: Error finding container b521ca27473fd48abc0235bbd9e4c48abeeaf34b297e084434b8f51dd25ef2fc: Status 404 returned error can't find the container with id b521ca27473fd48abc0235bbd9e4c48abeeaf34b297e084434b8f51dd25ef2fc Jul 06 14:01:45.674912 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.674803 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bdv7g" event={"ID":"ddafc31d-a61f-4e4d-87c9-d70e43da0637","Type":"ContainerStarted","Data":"3d0f113048d037ab5e1df359a316a6965c77688c2f4959c9935ee153d519012c"} Jul 06 14:01:45.676275 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.676250 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9d579" event={"ID":"691adf51-ea2f-4aca-aaea-c357787cac97","Type":"ContainerStarted","Data":"cec350a1584ac92549716472b9825efa14bf13e79a55f6d68013753b090f921e"} Jul 06 14:01:45.677940 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.677915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7kjp" event={"ID":"d2a56be5-b85c-4abb-828b-ea42f00a8f7d","Type":"ContainerStarted","Data":"bf7451bceb1cd6b5a4f0e644f2e9c8cb770c468acc5fa20ccb5fa93fcd6e1817"} Jul 06 14:01:45.678050 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.677963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7kjp" event={"ID":"d2a56be5-b85c-4abb-828b-ea42f00a8f7d","Type":"ContainerStarted","Data":"214f9635b3a0969c08db117233c3ca6ed96219130e0c46a65e8c732e41867a4f"} Jul 06 14:01:45.678050 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.678017 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:45.679034 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.679013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b5c88994-vsnlm" event={"ID":"348fadc9-b714-4bdc-983a-bfe26b9ebfac","Type":"ContainerStarted","Data":"b521ca27473fd48abc0235bbd9e4c48abeeaf34b297e084434b8f51dd25ef2fc"} Jul 06 14:01:45.680267 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.680247 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" event={"ID":"380d419c-c9e9-44fa-abb6-fab97c53fb75","Type":"ContainerStarted","Data":"b4ea4b4b346c11479b366d8ff6e9b2053d718750c0ec4c8ee6a03b9a8a205a8f"} Jul 06 14:01:45.680459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.680441 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" Jul 06 14:01:45.685224 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.685204 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" Jul 06 14:01:45.695720 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.695681 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bdv7g" podStartSLOduration=1.05784404 podStartE2EDuration="6.695666091s" totalImagesPullingTime="5.637822051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:45.695488454 +0000 UTC m=+56.884537500" watchObservedRunningTime="2026-07-06 14:01:45.695666091 +0000 UTC m=+56.884715135" Jul 06 14:01:45.713732 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.713677 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-d7kjp" podStartSLOduration=1.348079234 podStartE2EDuration="6.713661052s" totalImagesPullingTime="5.365581818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:45.713369077 +0000 UTC m=+56.902418123" watchObservedRunningTime="2026-07-06 14:01:45.713661052 +0000 UTC m=+56.902710099" Jul 06 14:01:45.732085 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:45.732033 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-gdmkp" podStartSLOduration=2.828575819 podStartE2EDuration="4.732018038s" totalImagesPullingTime="1.903442219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:45.730837159 +0000 UTC m=+56.919886206" watchObservedRunningTime="2026-07-06 14:01:45.732018038 +0000 UTC m=+56.921067062" Jul 06 14:01:46.211300 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.211263 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-b688k"] Jul 06 14:01:46.213551 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.213535 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.217853 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.217829 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 14:01:46.218008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.217861 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dlbt4\"" Jul 06 14:01:46.218008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.217873 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 14:01:46.218008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.217864 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 14:01:46.218008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.217929 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 14:01:46.218008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.217890 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 14:01:46.223948 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.223925 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-b688k"] Jul 06 14:01:46.258479 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.258439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc698a30-3594-49d6-830a-97122a99f6aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.258820 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.258521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc698a30-3594-49d6-830a-97122a99f6aa-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.258820 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.258562 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc698a30-3594-49d6-830a-97122a99f6aa-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.258820 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.258596 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tndsl\" (UniqueName: \"kubernetes.io/projected/cc698a30-3594-49d6-830a-97122a99f6aa-kube-api-access-tndsl\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.359865 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.359766 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc698a30-3594-49d6-830a-97122a99f6aa-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.359865 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.359818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tndsl\" (UniqueName: \"kubernetes.io/projected/cc698a30-3594-49d6-830a-97122a99f6aa-kube-api-access-tndsl\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.360097 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.359918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc698a30-3594-49d6-830a-97122a99f6aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.360097 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.360010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc698a30-3594-49d6-830a-97122a99f6aa-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.360678 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.360608 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc698a30-3594-49d6-830a-97122a99f6aa-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.362800 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.362766 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc698a30-3594-49d6-830a-97122a99f6aa-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.362800 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.362782 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc698a30-3594-49d6-830a-97122a99f6aa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.370501 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.370476 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tndsl\" (UniqueName: \"kubernetes.io/projected/cc698a30-3594-49d6-830a-97122a99f6aa-kube-api-access-tndsl\") pod \"prometheus-operator-567f7b8b7c-b688k\" (UID: \"cc698a30-3594-49d6-830a-97122a99f6aa\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.522517 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.522486 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" Jul 06 14:01:46.682519 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.682270 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-b688k"] Jul 06 14:01:46.686013 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:46.685981 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc698a30_3594_49d6_830a_97122a99f6aa.slice/crio-006555ee38af05602d1c706a35b4a4baf7125259d98b5870511c80819461deb4 WatchSource:0}: Error finding container 006555ee38af05602d1c706a35b4a4baf7125259d98b5870511c80819461deb4: Status 404 returned error can't find the container with id 006555ee38af05602d1c706a35b4a4baf7125259d98b5870511c80819461deb4 Jul 06 14:01:46.688844 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:46.688672 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9d579" event={"ID":"691adf51-ea2f-4aca-aaea-c357787cac97","Type":"ContainerStarted","Data":"5f0eedb95160176ac60bfbb65745043c7b640254da5fe4220fa19490edb7a1d0"} Jul 06 14:01:47.695933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:47.695876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" event={"ID":"cc698a30-3594-49d6-830a-97122a99f6aa","Type":"ContainerStarted","Data":"006555ee38af05602d1c706a35b4a4baf7125259d98b5870511c80819461deb4"} Jul 06 14:01:48.331159 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.331115 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c4f4f4b8-tkn78"] Jul 06 14:01:48.337050 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.337019 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.352422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.352388 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 14:01:48.353443 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.353413 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c4f4f4b8-tkn78"] Jul 06 14:01:48.374954 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.374923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-service-ca\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.375150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.374975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-oauth-config\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.375150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.375012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94cfb\" (UniqueName: \"kubernetes.io/projected/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-kube-api-access-94cfb\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.375150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.375054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-config\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.375150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.375079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-oauth-serving-cert\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.375150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.375102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-serving-cert\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.375345 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.375216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-trusted-ca-bundle\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.475590 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.475552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-oauth-config\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.475590 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.475595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94cfb\" (UniqueName: \"kubernetes.io/projected/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-kube-api-access-94cfb\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.475998 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.475636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-config\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.475998 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.475663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-oauth-serving-cert\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.475998 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.475688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-serving-cert\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.475998 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.475765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-trusted-ca-bundle\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.475998 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.475800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-service-ca\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.476489 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.476459 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-config\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.476609 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.476461 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-oauth-serving-cert\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.476778 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.476751 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-service-ca\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.477534 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.477503 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-trusted-ca-bundle\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.478561 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.478526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-oauth-config\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.478796 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.478774 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-serving-cert\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.486340 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.486310 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94cfb\" (UniqueName: \"kubernetes.io/projected/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-kube-api-access-94cfb\") pod \"console-78c4f4f4b8-tkn78\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:48.592073 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.591996 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vcfvm" Jul 06 14:01:48.649269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:48.649232 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:50.581073 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.580718 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c4f4f4b8-tkn78"] Jul 06 14:01:50.584435 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:50.584402 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ce89a4_f3cf_44f4_bc5f_e4ed6f48eb81.slice/crio-e81c0fc39e84bf7498dce6351cc7380f1a94468fc9b84d0e59f302cd5794594f WatchSource:0}: Error finding container e81c0fc39e84bf7498dce6351cc7380f1a94468fc9b84d0e59f302cd5794594f: Status 404 returned error can't find the container with id e81c0fc39e84bf7498dce6351cc7380f1a94468fc9b84d0e59f302cd5794594f Jul 06 14:01:50.711023 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.710866 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9d579" event={"ID":"691adf51-ea2f-4aca-aaea-c357787cac97","Type":"ContainerStarted","Data":"ad84b06710155d49d8808e59bb99d171e118062c8c39fbeae54b5f35d0d18420"} Jul 06 14:01:50.713355 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.713320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" event={"ID":"cc698a30-3594-49d6-830a-97122a99f6aa","Type":"ContainerStarted","Data":"cefa5336f17d8b7da68ffc63440050750a28946cf1222755383b3ba964a35944"} Jul 06 14:01:50.713511 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.713389 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" event={"ID":"cc698a30-3594-49d6-830a-97122a99f6aa","Type":"ContainerStarted","Data":"038d5c0b7e9efd401a6e16caf7f13134c9796d2ba221e9b98f4d9b2178c43c3d"} Jul 06 14:01:50.715054 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.715025 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b5c88994-vsnlm" event={"ID":"348fadc9-b714-4bdc-983a-bfe26b9ebfac","Type":"ContainerStarted","Data":"5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41"} Jul 06 14:01:50.717170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.717131 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c4f4f4b8-tkn78" event={"ID":"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81","Type":"ContainerStarted","Data":"1ebe99f95ca5ac34a879e24424734155fc66d547c2bcbb69c4165759f2714caa"} Jul 06 14:01:50.717304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.717174 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c4f4f4b8-tkn78" event={"ID":"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81","Type":"ContainerStarted","Data":"e81c0fc39e84bf7498dce6351cc7380f1a94468fc9b84d0e59f302cd5794594f"} Jul 06 14:01:50.740746 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.740680 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9d579" podStartSLOduration=1.165920179 podStartE2EDuration="11.740657103s" totalImagesPullingTime="10.574736924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:01:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:50.73931955 +0000 UTC m=+61.928368600" watchObservedRunningTime="2026-07-06 14:01:50.740657103 +0000 UTC m=+61.929706151" Jul 06 14:01:50.758329 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.758272 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-b688k" podStartSLOduration=1.010973997 podStartE2EDuration="4.758254138s" totalImagesPullingTime="3.747280141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:50.757793792 +0000 UTC m=+61.946842867" watchObservedRunningTime="2026-07-06 14:01:50.758254138 +0000 UTC m=+61.947303164" Jul 06 14:01:50.786700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.786642 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56b5c88994-vsnlm" podStartSLOduration=4.7344096780000005 podStartE2EDuration="9.786627804s" totalImagesPullingTime="5.052218126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:50.786390851 +0000 UTC m=+61.975439920" watchObservedRunningTime="2026-07-06 14:01:50.786627804 +0000 UTC m=+61.975676850" Jul 06 14:01:50.810518 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:50.810468 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78c4f4f4b8-tkn78" podStartSLOduration=2.810450972 podStartE2EDuration="2.810450972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:50.809126406 +0000 UTC m=+61.998175452" watchObservedRunningTime="2026-07-06 14:01:50.810450972 +0000 UTC m=+61.999500017" Jul 06 14:01:51.676779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:51.676728 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:51.677197 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:51.676804 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:51.681536 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:51.681512 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:51.726011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:51.725983 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:01:52.564652 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.564617 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b"] Jul 06 14:01:52.567283 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.567265 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.570359 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.570332 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 14:01:52.570491 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.570343 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-86nqv\"" Jul 06 14:01:52.570491 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.570409 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 14:01:52.577736 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.577712 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz"] Jul 06 14:01:52.579863 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.579846 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.581462 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.581444 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b"] Jul 06 14:01:52.583082 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.583048 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 14:01:52.583082 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.583068 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 14:01:52.583244 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.583048 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 14:01:52.583244 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.583158 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9xzcw\"" Jul 06 14:01:52.589752 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.589734 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-82zbt"] Jul 06 14:01:52.592063 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.592041 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.593631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.593608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz"] Jul 06 14:01:52.594774 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.594750 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 14:01:52.594911 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.594837 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 14:01:52.594911 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.594892 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 14:01:52.595031 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.594974 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gmjzc\"" Jul 06 14:01:52.608980 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.608953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.609129 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.608994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-wtmp\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609129 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609040 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.609129 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnj7p\" (UniqueName: \"kubernetes.io/projected/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-api-access-xnj7p\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.609292 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ls4r\" (UniqueName: \"kubernetes.io/projected/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-kube-api-access-5ls4r\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.609292 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dc19b82f-3557-445e-bcd7-33766be1a5cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.609292 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609231 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.609292 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609259 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.609292 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609282 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z55w9\" (UniqueName: \"kubernetes.io/projected/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-kube-api-access-z55w9\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609524 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609327 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-sys\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609524 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609359 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-tls\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609524 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.609524 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609524 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.609524 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609513 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609752 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc19b82f-3557-445e-bcd7-33766be1a5cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.609752 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-root\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609752 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-textfile\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.609752 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.609624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-metrics-client-ca\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.710556 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.710556 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc19b82f-3557-445e-bcd7-33766be1a5cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-root\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710596 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-textfile\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-metrics-client-ca\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710678 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-root\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:52.710712 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-wtmp\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:52.710763 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-tls podName:dc19b82f-3557-445e-bcd7-33766be1a5cc nodeName:}" failed. No retries permitted until 2026-07-06 14:01:53.210745936 +0000 UTC m=+64.399794967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-9d6vz" (UID: "dc19b82f-3557-445e-bcd7-33766be1a5cc") : secret "kube-state-metrics-tls" not found Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnj7p\" (UniqueName: \"kubernetes.io/projected/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-api-access-xnj7p\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-wtmp\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ls4r\" (UniqueName: \"kubernetes.io/projected/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-kube-api-access-5ls4r\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dc19b82f-3557-445e-bcd7-33766be1a5cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.710979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711028 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-textfile\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711126 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:52.711135 2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:52.711183 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-tls podName:dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb nodeName:}" failed. No retries permitted until 2026-07-06 14:01:53.211165621 +0000 UTC m=+64.400214651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-68h6b" (UID: "dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb") : secret "openshift-state-metrics-tls" not found Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z55w9\" (UniqueName: \"kubernetes.io/projected/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-kube-api-access-z55w9\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-sys\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-tls\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711302 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-sys\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711308 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-metrics-client-ca\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:52.711397 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:52.711445 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-tls podName:81ffeb38-a7fb-49a5-a723-21a8a5ed50fa nodeName:}" failed. No retries permitted until 2026-07-06 14:01:53.211430605 +0000 UTC m=+64.400479629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-tls") pod "node-exporter-82zbt" (UID: "81ffeb38-a7fb-49a5-a723-21a8a5ed50fa") : secret "node-exporter-tls" not found Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711454 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dc19b82f-3557-445e-bcd7-33766be1a5cc-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711545 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc19b82f-3557-445e-bcd7-33766be1a5cc-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711549 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.711936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711890 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.712697 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.711970 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.713558 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.713530 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.713746 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.713723 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:52.714018 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.714000 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.721975 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.721950 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnj7p\" (UniqueName: \"kubernetes.io/projected/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-api-access-xnj7p\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:52.722087 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.721998 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ls4r\" (UniqueName: \"kubernetes.io/projected/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-kube-api-access-5ls4r\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:52.723318 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:52.723295 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z55w9\" (UniqueName: \"kubernetes.io/projected/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-kube-api-access-z55w9\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:53.214978 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.214934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:53.214978 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.214979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-tls\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:53.215247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.215022 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:53.217349 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.217317 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/81ffeb38-a7fb-49a5-a723-21a8a5ed50fa-node-exporter-tls\") pod \"node-exporter-82zbt\" (UID: \"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa\") " pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:53.217465 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.217446 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dc19b82f-3557-445e-bcd7-33766be1a5cc-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-9d6vz\" (UID: \"dc19b82f-3557-445e-bcd7-33766be1a5cc\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:53.217537 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.217514 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-68h6b\" (UID: \"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:53.476259 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.476148 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" Jul 06 14:01:53.489123 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.489063 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" Jul 06 14:01:53.501004 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.500975 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-82zbt" Jul 06 14:01:53.513012 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:53.512777 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ffeb38_a7fb_49a5_a723_21a8a5ed50fa.slice/crio-0a128e29ff39a818a429150b55093f8fc0489f251503906e4ccfa4e264a9e6d5 WatchSource:0}: Error finding container 0a128e29ff39a818a429150b55093f8fc0489f251503906e4ccfa4e264a9e6d5: Status 404 returned error can't find the container with id 0a128e29ff39a818a429150b55093f8fc0489f251503906e4ccfa4e264a9e6d5 Jul 06 14:01:53.646243 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.645024 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b"] Jul 06 14:01:53.649350 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:53.649315 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd54d49_dc01_4f0b_b7f3_56eb2356a5cb.slice/crio-f7918a7530c00af6cc78213d55de984051bf63bcb67b20481fceb7ae0fe489a3 WatchSource:0}: Error finding container f7918a7530c00af6cc78213d55de984051bf63bcb67b20481fceb7ae0fe489a3: Status 404 returned error can't find the container with id f7918a7530c00af6cc78213d55de984051bf63bcb67b20481fceb7ae0fe489a3 Jul 06 14:01:53.655166 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.655145 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:01:53.657805 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.657784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.660842 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.660822 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 14:01:53.661157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.660838 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 14:01:53.661157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.660847 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 14:01:53.661157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.660881 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 14:01:53.661670 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.661560 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 14:01:53.661670 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.661619 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 14:01:53.661812 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.661623 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 14:01:53.664042 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.664018 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 14:01:53.664147 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.664101 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 14:01:53.664225 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.664206 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t45qr\"" Jul 06 14:01:53.672082 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.672056 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz"] Jul 06 14:01:53.676217 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.676197 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:01:53.679052 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:53.679030 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc19b82f_3557_445e_bcd7_33766be1a5cc.slice/crio-dcfc38529d7378ca504f490bb0b1821c18321dae33083df83b98ea491e8e551d WatchSource:0}: Error finding container dcfc38529d7378ca504f490bb0b1821c18321dae33083df83b98ea491e8e551d: Status 404 returned error can't find the container with id dcfc38529d7378ca504f490bb0b1821c18321dae33083df83b98ea491e8e551d Jul 06 14:01:53.718230 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718205 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718307 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-web-config\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718379 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4mnt\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-kube-api-access-x4mnt\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718396 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718412 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718476 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-out\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718877 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718555 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718877 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-volume\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.718877 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.718658 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.736178 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.736119 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" event={"ID":"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb","Type":"ContainerStarted","Data":"056f38025e1b6b5cf05b7593b56dffb20f774a36b19177d9cad435bacf853a7f"} Jul 06 14:01:53.736326 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.736241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" event={"ID":"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb","Type":"ContainerStarted","Data":"f7918a7530c00af6cc78213d55de984051bf63bcb67b20481fceb7ae0fe489a3"} Jul 06 14:01:53.737597 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.737568 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" event={"ID":"dc19b82f-3557-445e-bcd7-33766be1a5cc","Type":"ContainerStarted","Data":"dcfc38529d7378ca504f490bb0b1821c18321dae33083df83b98ea491e8e551d"} Jul 06 14:01:53.738879 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.738854 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-82zbt" event={"ID":"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa","Type":"ContainerStarted","Data":"0a128e29ff39a818a429150b55093f8fc0489f251503906e4ccfa4e264a9e6d5"} Jul 06 14:01:53.819598 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.819567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.819767 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.819619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-volume\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.819767 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.819733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.819875 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.819826 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.819951 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.819878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.819997 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.819946 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.819997 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.819976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820082 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-web-config\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820082 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4mnt\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-kube-api-access-x4mnt\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820082 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820230 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820230 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820341 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-out\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820341 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820256 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820432 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:53.820355 2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 14:01:53.820432 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:53.820412 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls podName:3bff8145-d4d5-441c-af7d-7bd8e3309f7c nodeName:}" failed. No retries permitted until 2026-07-06 14:01:54.32039454 +0000 UTC m=+65.509443564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c") : secret "alertmanager-main-tls" not found Jul 06 14:01:53.820624 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.820916 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.820870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.822934 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.822848 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-out\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.823107 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.823077 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-volume\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.823634 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.823455 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.823634 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.823510 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.823634 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.823579 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.823853 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.823729 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.824084 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.824064 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-web-config\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.824173 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.824146 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:53.829443 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:53.829420 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4mnt\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-kube-api-access-x4mnt\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:54.324616 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:54.324575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:54.327152 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:54.327118 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:54.568749 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:54.568505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:01:54.747034 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:54.746993 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" event={"ID":"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb","Type":"ContainerStarted","Data":"b9cf50610eabec738129c67e20e1e2a82d15697ec705ba321c7ee95369121dd9"} Jul 06 14:01:54.794210 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:54.794023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:01:54.929645 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:54.929578 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bff8145_d4d5_441c_af7d_7bd8e3309f7c.slice/crio-451ef1c6b3b065eb46c7824aa7c2c75a5ea4e927003822735b7b27eb26fc6419 WatchSource:0}: Error finding container 451ef1c6b3b065eb46c7824aa7c2c75a5ea4e927003822735b7b27eb26fc6419: Status 404 returned error can't find the container with id 451ef1c6b3b065eb46c7824aa7c2c75a5ea4e927003822735b7b27eb26fc6419 Jul 06 14:01:55.031962 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.031923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:55.034938 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.034911 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:01:55.044748 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.044720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09458f3b-e5f7-4f3d-b621-5396d4176382-metrics-certs\") pod \"network-metrics-daemon-ll8js\" (UID: \"09458f3b-e5f7-4f3d-b621-5396d4176382\") " pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:55.132535 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.132491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:55.132535 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.132543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:55.136003 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.135968 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:01:55.136003 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.135968 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:01:55.146427 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.146396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f31c9785-76aa-4397-8383-3d5d06adc6b8-original-pull-secret\") pod \"global-pull-secret-syncer-9g6dg\" (UID: \"f31c9785-76aa-4397-8383-3d5d06adc6b8\") " pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:55.146988 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.146962 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:01:55.156491 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.156468 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djdj5\" (UniqueName: \"kubernetes.io/projected/54f69821-e786-41ef-a3bf-41d31c11e2e2-kube-api-access-djdj5\") pod \"network-check-target-2rn4s\" (UID: \"54f69821-e786-41ef-a3bf-41d31c11e2e2\") " pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:55.294487 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.294465 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hqzzg\"" Jul 06 14:01:55.302117 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.302095 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ll8js" Jul 06 14:01:55.399163 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.399026 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9g6dg" Jul 06 14:01:55.410774 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.410203 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q6brz\"" Jul 06 14:01:55.416329 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.416114 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:01:55.477121 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.475753 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ll8js"] Jul 06 14:01:55.573616 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.573167 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v"] Jul 06 14:01:55.579212 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.578891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.584142 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.583152 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 14:01:55.584142 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.583957 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7u1o5g7rc80eb\"" Jul 06 14:01:55.585654 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.585244 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kndfk\"" Jul 06 14:01:55.585654 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.585264 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 14:01:55.585654 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.585481 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 14:01:55.585654 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.585522 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 14:01:55.585981 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.585708 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 14:01:55.596015 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.595956 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v"] Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-tls\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9g6dg"] Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-grpc-tls\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640330 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzxb9\" (UniqueName: \"kubernetes.io/projected/4aed4c15-624e-4cc8-a125-9c89b331c50f-kube-api-access-gzxb9\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.640828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.640406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aed4c15-624e-4cc8-a125-9c89b331c50f-metrics-client-ca\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.644983 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:55.644952 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31c9785_76aa_4397_8383_3d5d06adc6b8.slice/crio-84d1230aa6fa494d3a2fd50fef0f2173cca26143666c17c3b3654d83d3b4094c WatchSource:0}: Error finding container 84d1230aa6fa494d3a2fd50fef0f2173cca26143666c17c3b3654d83d3b4094c: Status 404 returned error can't find the container with id 84d1230aa6fa494d3a2fd50fef0f2173cca26143666c17c3b3654d83d3b4094c Jul 06 14:01:55.652885 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.652858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2rn4s"] Jul 06 14:01:55.656500 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:55.656474 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f69821_e786_41ef_a3bf_41d31c11e2e2.slice/crio-98c9abac93830caf0ce66abedafd0996cdb0db7efde42237ec8e48f57073af90 WatchSource:0}: Error finding container 98c9abac93830caf0ce66abedafd0996cdb0db7efde42237ec8e48f57073af90: Status 404 returned error can't find the container with id 98c9abac93830caf0ce66abedafd0996cdb0db7efde42237ec8e48f57073af90 Jul 06 14:01:55.692053 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.692025 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-d7kjp" Jul 06 14:01:55.741148 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.741148 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.741148 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.741353 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.741353 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741215 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-grpc-tls\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.741353 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzxb9\" (UniqueName: \"kubernetes.io/projected/4aed4c15-624e-4cc8-a125-9c89b331c50f-kube-api-access-gzxb9\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.741353 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aed4c15-624e-4cc8-a125-9c89b331c50f-metrics-client-ca\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.741481 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.741368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-tls\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.743742 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.743026 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aed4c15-624e-4cc8-a125-9c89b331c50f-metrics-client-ca\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.749983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.744663 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.749983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.745834 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.749983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.746307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-tls\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.749983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.747810 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.750538 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.750410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.751194 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.751116 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4aed4c15-624e-4cc8-a125-9c89b331c50f-secret-grpc-tls\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.755181 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.755136 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzxb9\" (UniqueName: \"kubernetes.io/projected/4aed4c15-624e-4cc8-a125-9c89b331c50f-kube-api-access-gzxb9\") pod \"thanos-querier-6cddf75cd8-kjz2v\" (UID: \"4aed4c15-624e-4cc8-a125-9c89b331c50f\") " pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:55.757571 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.757508 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerStarted","Data":"451ef1c6b3b065eb46c7824aa7c2c75a5ea4e927003822735b7b27eb26fc6419"} Jul 06 14:01:55.760231 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.760156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" event={"ID":"dc19b82f-3557-445e-bcd7-33766be1a5cc","Type":"ContainerStarted","Data":"3542e5098d64c62d2ad4e4cd8f24c51b4e6a7b316c605f42fca9fa2292bd8d91"} Jul 06 14:01:55.760231 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.760217 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" event={"ID":"dc19b82f-3557-445e-bcd7-33766be1a5cc","Type":"ContainerStarted","Data":"7c297a20d3797023465f77e1c970ea989e28c5303a79ef205b2c42e106886c8d"} Jul 06 14:01:55.760389 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.760246 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" event={"ID":"dc19b82f-3557-445e-bcd7-33766be1a5cc","Type":"ContainerStarted","Data":"5353cb000d52361546b42db5a027cb5f013a8d2242d8f1db67c7c28381189ea0"} Jul 06 14:01:55.762497 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.762195 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9g6dg" event={"ID":"f31c9785-76aa-4397-8383-3d5d06adc6b8","Type":"ContainerStarted","Data":"84d1230aa6fa494d3a2fd50fef0f2173cca26143666c17c3b3654d83d3b4094c"} Jul 06 14:01:55.763643 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.763617 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ll8js" event={"ID":"09458f3b-e5f7-4f3d-b621-5396d4176382","Type":"ContainerStarted","Data":"da9660b3b4ade69409956fea0fc9e8041f47961db4cff7f9e5f0834f6b139a6a"} Jul 06 14:01:55.766589 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.766535 2562 generic.go:356] "Generic (PLEG): container finished" podID="81ffeb38-a7fb-49a5-a723-21a8a5ed50fa" containerID="76117892005dbf9cef4b281bfe78e8955d23914a4f2d7ff56c3460fa80036882" exitCode=0 Jul 06 14:01:55.766740 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.766723 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-82zbt" event={"ID":"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa","Type":"ContainerDied","Data":"76117892005dbf9cef4b281bfe78e8955d23914a4f2d7ff56c3460fa80036882"} Jul 06 14:01:55.768949 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.768922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2rn4s" event={"ID":"54f69821-e786-41ef-a3bf-41d31c11e2e2","Type":"ContainerStarted","Data":"98c9abac93830caf0ce66abedafd0996cdb0db7efde42237ec8e48f57073af90"} Jul 06 14:01:55.771564 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.771535 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" event={"ID":"dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb","Type":"ContainerStarted","Data":"e06cffc066d09eccc781e01d37cac2c6bef9381ddd92d8cdfa7ea5e543035a3c"} Jul 06 14:01:55.783601 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.783531 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-9d6vz" podStartSLOduration=2.193895478 podStartE2EDuration="3.783511283s" totalImagesPullingTime="1.589615805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:55.78197537 +0000 UTC m=+66.971024441" watchObservedRunningTime="2026-07-06 14:01:55.783511283 +0000 UTC m=+66.972560333" Jul 06 14:01:55.825061 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.825004 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-68h6b" podStartSLOduration=2.337110557 podStartE2EDuration="3.824980581s" totalImagesPullingTime="1.487870024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:55.823643496 +0000 UTC m=+67.012692571" watchObservedRunningTime="2026-07-06 14:01:55.824980581 +0000 UTC m=+67.014029629" Jul 06 14:01:55.900780 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:55.900758 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:01:56.318988 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.318370 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v"] Jul 06 14:01:56.559097 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:56.559052 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aed4c15_624e_4cc8_a125_9c89b331c50f.slice/crio-b04b8c8cae0d106eb212ef189e662d29937e5608be7542d3ca8963eb7f61975e WatchSource:0}: Error finding container b04b8c8cae0d106eb212ef189e662d29937e5608be7542d3ca8963eb7f61975e: Status 404 returned error can't find the container with id b04b8c8cae0d106eb212ef189e662d29937e5608be7542d3ca8963eb7f61975e Jul 06 14:01:56.779867 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.779831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ll8js" event={"ID":"09458f3b-e5f7-4f3d-b621-5396d4176382","Type":"ContainerStarted","Data":"deebf7fd52e0dc7a7f1a0a0d6380e039af5faf0b076b627a0cf77a5e6bf3f4d2"} Jul 06 14:01:56.783475 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.783308 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-82zbt" event={"ID":"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa","Type":"ContainerStarted","Data":"c3963ababdca5d62f207a27c0f113c827edd751fc6be5ad0d7c4874ad99118e8"} Jul 06 14:01:56.783475 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.783373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-82zbt" event={"ID":"81ffeb38-a7fb-49a5-a723-21a8a5ed50fa","Type":"ContainerStarted","Data":"af82c8b8940010c05059f21a83b2817ba8b3300d813c577f62e268ca8a6a8be0"} Jul 06 14:01:56.785921 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.785879 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" event={"ID":"4aed4c15-624e-4cc8-a125-9c89b331c50f","Type":"ContainerStarted","Data":"b04b8c8cae0d106eb212ef189e662d29937e5608be7542d3ca8963eb7f61975e"} Jul 06 14:01:56.787991 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.787962 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerID="7613e0076de8dc9cee12dc87f5acc66ccafab755b0e22cd9f31f37ea7e2aeebe" exitCode=0 Jul 06 14:01:56.788309 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.788257 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"7613e0076de8dc9cee12dc87f5acc66ccafab755b0e22cd9f31f37ea7e2aeebe"} Jul 06 14:01:56.812361 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.812250 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-82zbt" podStartSLOduration=3.583603846 podStartE2EDuration="4.812226469s" totalImagesPullingTime="1.146450296s" totalInitContainerRuntime="82.172327ms" isStatefulPod=true podCreationTimestamp="2026-07-06 14:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:56.810663856 +0000 UTC m=+67.999712930" watchObservedRunningTime="2026-07-06 14:01:56.812226469 +0000 UTC m=+68.001275568" Jul 06 14:01:56.907029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.906995 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-cc4cccc59-rvbwb"] Jul 06 14:01:56.909949 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.909891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:56.913143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.913036 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 14:01:56.913143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.913074 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 14:01:56.913347 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.913074 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-daei1gql08qgh\"" Jul 06 14:01:56.913400 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.913363 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 14:01:56.913449 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.913422 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 14:01:56.913449 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.913433 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dzc8c\"" Jul 06 14:01:56.920832 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.920769 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-cc4cccc59-rvbwb"] Jul 06 14:01:56.953412 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.953346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-client-ca-bundle\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:56.953669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.953478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bd29ce12-fb44-4780-ad0b-638ced353e4b-metrics-server-audit-profiles\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:56.953669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.953513 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bd29ce12-fb44-4780-ad0b-638ced353e4b-audit-log\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:56.953669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.953552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd29ce12-fb44-4780-ad0b-638ced353e4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:56.953669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.953580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-secret-metrics-server-client-certs\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:56.953669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.953602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2xj2\" (UniqueName: \"kubernetes.io/projected/bd29ce12-fb44-4780-ad0b-638ced353e4b-kube-api-access-b2xj2\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:56.953669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:56.953653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-secret-metrics-server-tls\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.054775 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.054733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bd29ce12-fb44-4780-ad0b-638ced353e4b-metrics-server-audit-profiles\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.055025 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.054784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bd29ce12-fb44-4780-ad0b-638ced353e4b-audit-log\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.055025 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.054831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd29ce12-fb44-4780-ad0b-638ced353e4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.055025 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.054852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-secret-metrics-server-client-certs\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.055025 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.054867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2xj2\" (UniqueName: \"kubernetes.io/projected/bd29ce12-fb44-4780-ad0b-638ced353e4b-kube-api-access-b2xj2\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.055025 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.054949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-secret-metrics-server-tls\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.055025 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.054994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-client-ca-bundle\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.055695 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.055651 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd29ce12-fb44-4780-ad0b-638ced353e4b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.057667 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.057336 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bd29ce12-fb44-4780-ad0b-638ced353e4b-metrics-server-audit-profiles\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.057667 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.057616 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bd29ce12-fb44-4780-ad0b-638ced353e4b-audit-log\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.058990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.058368 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-secret-metrics-server-client-certs\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.058990 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.058414 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-secret-metrics-server-tls\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.065759 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.065669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd29ce12-fb44-4780-ad0b-638ced353e4b-client-ca-bundle\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.067312 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.067287 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2xj2\" (UniqueName: \"kubernetes.io/projected/bd29ce12-fb44-4780-ad0b-638ced353e4b-kube-api-access-b2xj2\") pod \"metrics-server-cc4cccc59-rvbwb\" (UID: \"bd29ce12-fb44-4780-ad0b-638ced353e4b\") " pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.223844 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.223551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:01:57.349262 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.349230 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8"] Jul 06 14:01:57.354206 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.353420 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:57.357739 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.357698 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nb2vj\"" Jul 06 14:01:57.360936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.360285 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 14:01:57.363648 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.363476 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8"] Jul 06 14:01:57.408265 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.408181 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-cc4cccc59-rvbwb"] Jul 06 14:01:57.412077 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:01:57.412042 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd29ce12_fb44_4780_ad0b_638ced353e4b.slice/crio-f982309d20d7336ed4fecd50ec3bd44419ff0c59e33fcd864c8b0ece90324c70 WatchSource:0}: Error finding container f982309d20d7336ed4fecd50ec3bd44419ff0c59e33fcd864c8b0ece90324c70: Status 404 returned error can't find the container with id f982309d20d7336ed4fecd50ec3bd44419ff0c59e33fcd864c8b0ece90324c70 Jul 06 14:01:57.459574 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.459374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w298q\" (UniqueName: \"kubernetes.io/projected/9df69d67-9d32-4bc1-b99e-3891f527d517-kube-api-access-w298q\") pod \"monitoring-plugin-d5dcff7cf-6hlg8\" (UID: \"9df69d67-9d32-4bc1-b99e-3891f527d517\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:57.459574 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.459463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9df69d67-9d32-4bc1-b99e-3891f527d517-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6hlg8\" (UID: \"9df69d67-9d32-4bc1-b99e-3891f527d517\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:57.560560 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.560016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w298q\" (UniqueName: \"kubernetes.io/projected/9df69d67-9d32-4bc1-b99e-3891f527d517-kube-api-access-w298q\") pod \"monitoring-plugin-d5dcff7cf-6hlg8\" (UID: \"9df69d67-9d32-4bc1-b99e-3891f527d517\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:57.560560 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.560096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9df69d67-9d32-4bc1-b99e-3891f527d517-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6hlg8\" (UID: \"9df69d67-9d32-4bc1-b99e-3891f527d517\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:57.560560 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:57.560288 2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 06 14:01:57.560560 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:01:57.560355 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9df69d67-9d32-4bc1-b99e-3891f527d517-monitoring-plugin-cert podName:9df69d67-9d32-4bc1-b99e-3891f527d517 nodeName:}" failed. No retries permitted until 2026-07-06 14:01:58.060331226 +0000 UTC m=+69.249380254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9df69d67-9d32-4bc1-b99e-3891f527d517-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-6hlg8" (UID: "9df69d67-9d32-4bc1-b99e-3891f527d517") : secret "monitoring-plugin-cert" not found Jul 06 14:01:57.571253 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.571191 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w298q\" (UniqueName: \"kubernetes.io/projected/9df69d67-9d32-4bc1-b99e-3891f527d517-kube-api-access-w298q\") pod \"monitoring-plugin-d5dcff7cf-6hlg8\" (UID: \"9df69d67-9d32-4bc1-b99e-3891f527d517\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:57.796304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.796256 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ll8js" event={"ID":"09458f3b-e5f7-4f3d-b621-5396d4176382","Type":"ContainerStarted","Data":"8ca9f87c69e21b9876987c6003720d6c453fa818ffbd92fa743bd79ede591454"} Jul 06 14:01:57.800238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:57.800179 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" event={"ID":"bd29ce12-fb44-4780-ad0b-638ced353e4b","Type":"ContainerStarted","Data":"f982309d20d7336ed4fecd50ec3bd44419ff0c59e33fcd864c8b0ece90324c70"} Jul 06 14:01:58.066530 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.065952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9df69d67-9d32-4bc1-b99e-3891f527d517-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6hlg8\" (UID: \"9df69d67-9d32-4bc1-b99e-3891f527d517\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:58.078170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.077076 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9df69d67-9d32-4bc1-b99e-3891f527d517-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6hlg8\" (UID: \"9df69d67-9d32-4bc1-b99e-3891f527d517\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:58.272275 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.272009 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:01:58.366668 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.366325 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ll8js" podStartSLOduration=68.270824626 podStartE2EDuration="1m9.366301959s" totalImagesPullingTime="1.095477333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:01:57.827991427 +0000 UTC m=+69.017040496" watchObservedRunningTime="2026-07-06 14:01:58.366301959 +0000 UTC m=+69.555351005" Jul 06 14:01:58.367344 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.367085 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c4f4f4b8-tkn78"] Jul 06 14:01:58.404297 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.402695 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-796bc448d6-p68v5"] Jul 06 14:01:58.405764 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.405719 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.421350 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.421324 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796bc448d6-p68v5"] Jul 06 14:01:58.469077 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.469041 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-config\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.469260 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.469095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-oauth-config\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.469260 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.469128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-oauth-serving-cert\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.469260 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.469181 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsk29\" (UniqueName: \"kubernetes.io/projected/26e6c50c-c876-4b2d-b63a-41cde6fb678b-kube-api-access-qsk29\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.469260 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.469232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-serving-cert\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.469459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.469286 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-service-ca\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.469459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.469320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-trusted-ca-bundle\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.570359 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.570320 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-service-ca\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.570541 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.570377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-trusted-ca-bundle\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.570541 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.570408 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-config\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.570541 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.570438 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-oauth-config\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.570541 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.570480 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-oauth-serving-cert\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.570758 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.570543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsk29\" (UniqueName: \"kubernetes.io/projected/26e6c50c-c876-4b2d-b63a-41cde6fb678b-kube-api-access-qsk29\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.570758 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.570599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-serving-cert\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.571911 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.571866 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-config\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.572437 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.572411 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-service-ca\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.572662 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.572617 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-oauth-serving-cert\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.572770 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.572675 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-trusted-ca-bundle\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.575418 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.575376 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-serving-cert\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.575972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.575931 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-oauth-config\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.590698 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.590669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsk29\" (UniqueName: \"kubernetes.io/projected/26e6c50c-c876-4b2d-b63a-41cde6fb678b-kube-api-access-qsk29\") pod \"console-796bc448d6-p68v5\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:58.649975 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.649656 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:01:58.721587 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:58.721545 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:01:59.005418 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.005214 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:01:59.008352 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.008324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.011969 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.011469 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 14:01:59.011969 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.011526 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 14:01:59.011969 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.011725 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a4kpp1o720qt5\"" Jul 06 14:01:59.011969 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.011753 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 14:01:59.012494 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.012475 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-q2vdj\"" Jul 06 14:01:59.013713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.013638 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 14:01:59.014046 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014024 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 14:01:59.015043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014189 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 14:01:59.015043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014039 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 14:01:59.015043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014091 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 14:01:59.015043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014074 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 14:01:59.015043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014119 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 14:01:59.015043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014052 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 14:01:59.015043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.014121 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 14:01:59.024025 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.023466 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 14:01:59.029390 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.029361 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:01:59.029690 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.029547 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 14:01:59.078504 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-config-out\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-config\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-web-config\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078745 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078761 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078794 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078874 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t8fq\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-kube-api-access-9t8fq\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.078984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.078973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.079424 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.079029 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.079424 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.079060 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.079424 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.079121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.079424 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.079152 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.079424 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.079191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180189 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180203 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-config-out\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-config\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180461 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180534 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-web-config\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.180644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181208 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181208 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180754 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181208 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t8fq\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-kube-api-access-9t8fq\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181208 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181208 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.180843 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181441 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.181391 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181490 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.181466 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.181956 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.181925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.182054 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.182000 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.185161 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.185111 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.186539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.186496 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.187852 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.186761 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-config-out\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.187852 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.187226 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.187852 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.187316 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.187852 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.187638 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.187852 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.187810 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.188174 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.188025 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.188444 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.188417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.188524 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.188424 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.189130 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.189096 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-config\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.189130 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.189110 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.189552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.189528 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.190822 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.190800 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.190958 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.190885 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-web-config\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.195505 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.195453 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t8fq\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-kube-api-access-9t8fq\") pod \"prometheus-k8s-0\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:01:59.328502 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:01:59.328444 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:02:02.754130 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.753892 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8"] Jul 06 14:02:02.811800 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.811769 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-796bc448d6-p68v5"] Jul 06 14:02:02.817225 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:02:02.817191 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e6c50c_c876_4b2d_b63a_41cde6fb678b.slice/crio-6bf99e1a8afc017a5f5407ae5da07aee3d1c2a2fea63879375c8f0096a90f4c5 WatchSource:0}: Error finding container 6bf99e1a8afc017a5f5407ae5da07aee3d1c2a2fea63879375c8f0096a90f4c5: Status 404 returned error can't find the container with id 6bf99e1a8afc017a5f5407ae5da07aee3d1c2a2fea63879375c8f0096a90f4c5 Jul 06 14:02:02.831390 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.831353 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" event={"ID":"9df69d67-9d32-4bc1-b99e-3891f527d517","Type":"ContainerStarted","Data":"a35d20492a471b568b38e3b9d09f4e6d71e802550d46c2161d8838c2ee08ad91"} Jul 06 14:02:02.842200 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.842131 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2rn4s" event={"ID":"54f69821-e786-41ef-a3bf-41d31c11e2e2","Type":"ContainerStarted","Data":"afa1cc5c7ef71406d2848a5d78b21d719ed69bf5b1f90f02e7ba7bfc646a06a3"} Jul 06 14:02:02.842426 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.842379 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:02:02.844854 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.844797 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" event={"ID":"4aed4c15-624e-4cc8-a125-9c89b331c50f","Type":"ContainerStarted","Data":"aaf9ab8a34e8a3e3c99c1e608ff70689dee6f9e9740ce26ddd14a34862c15e0f"} Jul 06 14:02:02.850724 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.849697 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:02:02.850724 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.849787 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bc448d6-p68v5" event={"ID":"26e6c50c-c876-4b2d-b63a-41cde6fb678b","Type":"ContainerStarted","Data":"6bf99e1a8afc017a5f5407ae5da07aee3d1c2a2fea63879375c8f0096a90f4c5"} Jul 06 14:02:02.854303 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:02:02.854254 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318f09bd_de33_4503_ad06_e7d31b7b8865.slice/crio-b9ec1fda8572b60879c552e103dd8d0ab390eeaa0e09faabbb78097de096f806 WatchSource:0}: Error finding container b9ec1fda8572b60879c552e103dd8d0ab390eeaa0e09faabbb78097de096f806: Status 404 returned error can't find the container with id b9ec1fda8572b60879c552e103dd8d0ab390eeaa0e09faabbb78097de096f806 Jul 06 14:02:02.855805 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.855699 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerStarted","Data":"e3c5588bc568d71c2a2fadb23094434edbc05b682d984602ddce30e669c40e97"} Jul 06 14:02:02.857784 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.857733 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" event={"ID":"bd29ce12-fb44-4780-ad0b-638ced353e4b","Type":"ContainerStarted","Data":"b6e5144cc389fbad2a5ea27f5c8df5d42473c74dfcc5ec8827c5a012e27da8ed"} Jul 06 14:02:02.868475 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.867213 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9g6dg" podStartSLOduration=66.907599789 podStartE2EDuration="1m13.867193635s" totalImagesPullingTime="6.959593846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:02.864228512 +0000 UTC m=+74.053277561" watchObservedRunningTime="2026-07-06 14:02:02.867193635 +0000 UTC m=+74.056242686" Jul 06 14:02:02.909391 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.909337 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" podStartSLOduration=1.741784584 podStartE2EDuration="6.909315031s" totalImagesPullingTime="5.167530447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:02.886622886 +0000 UTC m=+74.075671976" watchObservedRunningTime="2026-07-06 14:02:02.909315031 +0000 UTC m=+74.098364080" Jul 06 14:02:02.911385 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:02.911167 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2rn4s" podStartSLOduration=66.974792962 podStartE2EDuration="1m13.911151383s" totalImagesPullingTime="6.936358421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:02.91051501 +0000 UTC m=+74.099564082" watchObservedRunningTime="2026-07-06 14:02:02.911151383 +0000 UTC m=+74.100200432" Jul 06 14:02:03.866157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.866074 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerStarted","Data":"1086f00e1d68b097cc6f5338208ef7a05da507630e42b9a1af41c976783fc147"} Jul 06 14:02:03.866157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.866158 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerStarted","Data":"fe949c895700be13fa0dba57d4d74c6abe23b5fe78d6fca0e82ae5be07a47604"} Jul 06 14:02:03.866629 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.866201 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerStarted","Data":"313d7b5cb87334599120590d1c548bc8488abcd7b7c990028af28f7760bd528a"} Jul 06 14:02:03.866629 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.866250 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerStarted","Data":"b0b02aa4d08f251fcf3174b5cb063c4c70165eb400a0e456de3ea8159b39f84c"} Jul 06 14:02:03.867813 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.867786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9g6dg" event={"ID":"f31c9785-76aa-4397-8383-3d5d06adc6b8","Type":"ContainerStarted","Data":"ae465dc0d73757361933fd7232ccd5aa442c36919db6f75f42e9c06df200eeff"} Jul 06 14:02:03.870330 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.870301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" event={"ID":"4aed4c15-624e-4cc8-a125-9c89b331c50f","Type":"ContainerStarted","Data":"4691394a84c390637b036f521f3e181b480b9579f37ca2df0b4a300df3e135a4"} Jul 06 14:02:03.870448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.870367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" event={"ID":"4aed4c15-624e-4cc8-a125-9c89b331c50f","Type":"ContainerStarted","Data":"b1a21ec5238672d4febca6f6885a4a9fc445f5d42c15ee6fcbbcee75badf5bf7"} Jul 06 14:02:03.871876 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.871845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bc448d6-p68v5" event={"ID":"26e6c50c-c876-4b2d-b63a-41cde6fb678b","Type":"ContainerStarted","Data":"ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a"} Jul 06 14:02:03.873476 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.873455 2562 generic.go:356] "Generic (PLEG): container finished" podID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" exitCode=0 Jul 06 14:02:03.873574 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.873497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1"} Jul 06 14:02:03.873574 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.873558 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerStarted","Data":"b9ec1fda8572b60879c552e103dd8d0ab390eeaa0e09faabbb78097de096f806"} Jul 06 14:02:03.895191 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:03.895136 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-796bc448d6-p68v5" podStartSLOduration=5.895114562 podStartE2EDuration="5.895114562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:03.892588571 +0000 UTC m=+75.081637654" watchObservedRunningTime="2026-07-06 14:02:03.895114562 +0000 UTC m=+75.084163588" Jul 06 14:02:04.882724 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.882681 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerStarted","Data":"295ca3650f814f064f4aa76a19df95cca00ccae5b2ea6d6cb7f14f979bcb4333"} Jul 06 14:02:04.884674 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.884642 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" event={"ID":"9df69d67-9d32-4bc1-b99e-3891f527d517","Type":"ContainerStarted","Data":"5c53944f82c6ed58db97802c630b47474ed237a7df5129ae83f6d7bd596444ff"} Jul 06 14:02:04.884861 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.884826 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:02:04.888386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.888239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" event={"ID":"4aed4c15-624e-4cc8-a125-9c89b331c50f","Type":"ContainerStarted","Data":"6574fe03879ca423d4ec20f1fe6580ed248649c12289184c9340412facf2e132"} Jul 06 14:02:04.888386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.888326 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" event={"ID":"4aed4c15-624e-4cc8-a125-9c89b331c50f","Type":"ContainerStarted","Data":"5d03be37d1eb7bb1ec438748e26a6de79e737428473eea57c395f421203aea9c"} Jul 06 14:02:04.888386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.888368 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" event={"ID":"4aed4c15-624e-4cc8-a125-9c89b331c50f","Type":"ContainerStarted","Data":"5d9293a950d6dd738da4bb8eef96868b7d6dcb749bf2b20768b16d361ab43f72"} Jul 06 14:02:04.891021 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.890996 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" Jul 06 14:02:04.935028 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.934967 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.827308221 podStartE2EDuration="11.934945095s" totalImagesPullingTime="8.086667632s" totalInitContainerRuntime="20.969242ms" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:04.933219957 +0000 UTC m=+76.122269047" watchObservedRunningTime="2026-07-06 14:02:04.934945095 +0000 UTC m=+76.123994145" Jul 06 14:02:04.967541 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.967483 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" podStartSLOduration=2.728233928 podStartE2EDuration="9.96746715s" totalImagesPullingTime="7.239233222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:04.966470325 +0000 UTC m=+76.155519400" watchObservedRunningTime="2026-07-06 14:02:04.96746715 +0000 UTC m=+76.156516222" Jul 06 14:02:04.998102 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:04.998047 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6hlg8" podStartSLOduration=6.566132123 podStartE2EDuration="7.998028619s" totalImagesPullingTime="1.431896496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:04.997454887 +0000 UTC m=+76.186503954" watchObservedRunningTime="2026-07-06 14:02:04.998028619 +0000 UTC m=+76.187077666" Jul 06 14:02:05.893060 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:05.893017 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:02:07.906109 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:07.906075 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerStarted","Data":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} Jul 06 14:02:07.906467 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:07.906153 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerStarted","Data":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} Jul 06 14:02:07.906467 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:07.906195 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerStarted","Data":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} Jul 06 14:02:07.906467 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:07.906232 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerStarted","Data":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} Jul 06 14:02:07.906467 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:07.906270 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerStarted","Data":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} Jul 06 14:02:07.906467 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:07.906304 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerStarted","Data":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} Jul 06 14:02:07.947918 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:07.947829 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.836744319 podStartE2EDuration="9.947806179s" totalImagesPullingTime="3.08017529s" totalInitContainerRuntime="30.88657ms" isStatefulPod=false podCreationTimestamp="2026-07-06 14:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:02:07.945173804 +0000 UTC m=+79.134222873" watchObservedRunningTime="2026-07-06 14:02:07.947806179 +0000 UTC m=+79.136855227" Jul 06 14:02:08.722099 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:08.722057 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:02:08.722302 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:08.722144 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:02:08.726993 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:08.726968 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:02:08.913755 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:08.913724 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:02:08.968812 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:08.968779 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b5c88994-vsnlm"] Jul 06 14:02:09.329033 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:09.329003 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:02:11.903823 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:11.903795 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6cddf75cd8-kjz2v" Jul 06 14:02:17.224629 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:17.224590 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:02:17.225118 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:17.224699 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:02:23.399047 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:23.398998 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-78c4f4f4b8-tkn78" podUID="69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" containerName="console" containerID="cri-o://1ebe99f95ca5ac34a879e24424734155fc66d547c2bcbb69c4165759f2714caa" gracePeriod=15 Jul 06 14:02:23.977504 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:23.977471 2562 generic.go:356] "Generic (PLEG): container finished" podID="69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" containerID="1ebe99f95ca5ac34a879e24424734155fc66d547c2bcbb69c4165759f2714caa" exitCode=0 Jul 06 14:02:23.977669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:23.977516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c4f4f4b8-tkn78" event={"ID":"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81","Type":"ContainerDied","Data":"1ebe99f95ca5ac34a879e24424734155fc66d547c2bcbb69c4165759f2714caa"} Jul 06 14:02:24.177713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.177685 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:02:24.228304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228220 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-trusted-ca-bundle\") pod \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " Jul 06 14:02:24.228474 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228304 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-serving-cert\") pod \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " Jul 06 14:02:24.228474 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228342 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-service-ca\") pod \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " Jul 06 14:02:24.228474 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228363 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-oauth-config\") pod \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " Jul 06 14:02:24.228474 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228388 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-oauth-serving-cert\") pod \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " Jul 06 14:02:24.228474 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228431 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94cfb\" (UniqueName: \"kubernetes.io/projected/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-kube-api-access-94cfb\") pod \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " Jul 06 14:02:24.228692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228482 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-config\") pod \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\" (UID: \"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81\") " Jul 06 14:02:24.228736 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228681 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" (UID: "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:02:24.228779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.228749 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-service-ca" (OuterVolumeSpecName: "service-ca") pod "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" (UID: "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:02:24.229153 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.229134 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-trusted-ca-bundle\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:24.229251 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.229154 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-service-ca\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:24.229338 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.229313 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" (UID: "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:02:24.229628 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.229591 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-config" (OuterVolumeSpecName: "console-config") pod "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" (UID: "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:02:24.230884 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.230855 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" (UID: "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:02:24.231009 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.230970 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" (UID: "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:02:24.231075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.231032 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-kube-api-access-94cfb" (OuterVolumeSpecName: "kube-api-access-94cfb") pod "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" (UID: "69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81"). InnerVolumeSpecName "kube-api-access-94cfb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:02:24.330377 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.330343 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:24.330377 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.330372 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-oauth-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:24.330377 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.330382 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-oauth-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:24.330607 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.330390 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94cfb\" (UniqueName: \"kubernetes.io/projected/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-kube-api-access-94cfb\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:24.330607 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.330400 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81-console-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:24.982525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.982494 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c4f4f4b8-tkn78" Jul 06 14:02:24.982938 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.982494 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c4f4f4b8-tkn78" event={"ID":"69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81","Type":"ContainerDied","Data":"e81c0fc39e84bf7498dce6351cc7380f1a94468fc9b84d0e59f302cd5794594f"} Jul 06 14:02:24.982938 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:24.982624 2562 scope.go:117] "RemoveContainer" containerID="1ebe99f95ca5ac34a879e24424734155fc66d547c2bcbb69c4165759f2714caa" Jul 06 14:02:25.009632 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:25.009602 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c4f4f4b8-tkn78"] Jul 06 14:02:25.011001 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:25.010974 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78c4f4f4b8-tkn78"] Jul 06 14:02:25.388706 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:25.388662 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" path="/var/lib/kubelet/pods/69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81/volumes" Jul 06 14:02:33.876613 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:33.876580 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2rn4s" Jul 06 14:02:33.995830 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:33.995790 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56b5c88994-vsnlm" podUID="348fadc9-b714-4bdc-983a-bfe26b9ebfac" containerName="console" containerID="cri-o://5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41" gracePeriod=15 Jul 06 14:02:34.276233 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.276197 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:02:34.316453 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.316423 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-serving-cert\") pod \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " Jul 06 14:02:34.316666 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.316475 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-oauth-config\") pod \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " Jul 06 14:02:34.316666 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.316508 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-oauth-serving-cert\") pod \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " Jul 06 14:02:34.316666 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.316544 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-service-ca\") pod \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " Jul 06 14:02:34.316666 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.316575 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltbcr\" (UniqueName: \"kubernetes.io/projected/348fadc9-b714-4bdc-983a-bfe26b9ebfac-kube-api-access-ltbcr\") pod \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " Jul 06 14:02:34.316666 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.316640 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-config\") pod \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\" (UID: \"348fadc9-b714-4bdc-983a-bfe26b9ebfac\") " Jul 06 14:02:34.317013 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.316972 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-service-ca" (OuterVolumeSpecName: "service-ca") pod "348fadc9-b714-4bdc-983a-bfe26b9ebfac" (UID: "348fadc9-b714-4bdc-983a-bfe26b9ebfac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:02:34.317216 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.317194 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "348fadc9-b714-4bdc-983a-bfe26b9ebfac" (UID: "348fadc9-b714-4bdc-983a-bfe26b9ebfac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:02:34.317304 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.317208 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-config" (OuterVolumeSpecName: "console-config") pod "348fadc9-b714-4bdc-983a-bfe26b9ebfac" (UID: "348fadc9-b714-4bdc-983a-bfe26b9ebfac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:02:34.318966 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.318933 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/348fadc9-b714-4bdc-983a-bfe26b9ebfac-kube-api-access-ltbcr" (OuterVolumeSpecName: "kube-api-access-ltbcr") pod "348fadc9-b714-4bdc-983a-bfe26b9ebfac" (UID: "348fadc9-b714-4bdc-983a-bfe26b9ebfac"). InnerVolumeSpecName "kube-api-access-ltbcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:02:34.319277 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.319245 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "348fadc9-b714-4bdc-983a-bfe26b9ebfac" (UID: "348fadc9-b714-4bdc-983a-bfe26b9ebfac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:02:34.319370 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.319337 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "348fadc9-b714-4bdc-983a-bfe26b9ebfac" (UID: "348fadc9-b714-4bdc-983a-bfe26b9ebfac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:02:34.418247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.418208 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:34.418247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.418241 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:34.418247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.418251 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/348fadc9-b714-4bdc-983a-bfe26b9ebfac-console-oauth-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:34.418488 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.418260 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-oauth-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:34.418488 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.418269 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/348fadc9-b714-4bdc-983a-bfe26b9ebfac-service-ca\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:34.418488 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:34.418277 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltbcr\" (UniqueName: \"kubernetes.io/projected/348fadc9-b714-4bdc-983a-bfe26b9ebfac-kube-api-access-ltbcr\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:02:35.028317 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.028286 2562 generic.go:356] "Generic (PLEG): container finished" podID="348fadc9-b714-4bdc-983a-bfe26b9ebfac" containerID="5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41" exitCode=0 Jul 06 14:02:35.028713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.028329 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b5c88994-vsnlm" Jul 06 14:02:35.028713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.028371 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b5c88994-vsnlm" event={"ID":"348fadc9-b714-4bdc-983a-bfe26b9ebfac","Type":"ContainerDied","Data":"5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41"} Jul 06 14:02:35.028713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.028431 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b5c88994-vsnlm" event={"ID":"348fadc9-b714-4bdc-983a-bfe26b9ebfac","Type":"ContainerDied","Data":"b521ca27473fd48abc0235bbd9e4c48abeeaf34b297e084434b8f51dd25ef2fc"} Jul 06 14:02:35.028713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.028458 2562 scope.go:117] "RemoveContainer" containerID="5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41" Jul 06 14:02:35.039188 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.039169 2562 scope.go:117] "RemoveContainer" containerID="5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41" Jul 06 14:02:35.039511 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:02:35.039488 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41\": container with ID starting with 5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41 not found: ID does not exist" containerID="5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41" Jul 06 14:02:35.039560 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.039528 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41"} err="failed to get container status \"5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41\": rpc error: code = NotFound desc = could not find container \"5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41\": container with ID starting with 5e834edbe92a11aab97d1494c894173801e98c0d2e64144eb9e36a1f77fdee41 not found: ID does not exist" Jul 06 14:02:35.051627 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.051593 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b5c88994-vsnlm"] Jul 06 14:02:35.057535 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.057508 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56b5c88994-vsnlm"] Jul 06 14:02:35.388577 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:35.388500 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="348fadc9-b714-4bdc-983a-bfe26b9ebfac" path="/var/lib/kubelet/pods/348fadc9-b714-4bdc-983a-bfe26b9ebfac/volumes" Jul 06 14:02:37.230541 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:37.230508 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:02:37.235560 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:37.235536 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-cc4cccc59-rvbwb" Jul 06 14:02:59.329336 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:59.329304 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:02:59.349175 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:02:59.349150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:00.145381 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:00.145352 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:12.957579 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:12.957537 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:03:12.958254 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:12.957812 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="alertmanager" containerID="cri-o://e3c5588bc568d71c2a2fadb23094434edbc05b682d984602ddce30e669c40e97" gracePeriod=120 Jul 06 14:03:12.958254 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:12.957918 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-metric" containerID="cri-o://1086f00e1d68b097cc6f5338208ef7a05da507630e42b9a1af41c976783fc147" gracePeriod=120 Jul 06 14:03:12.958254 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:12.958000 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="config-reloader" containerID="cri-o://b0b02aa4d08f251fcf3174b5cb063c4c70165eb400a0e456de3ea8159b39f84c" gracePeriod=120 Jul 06 14:03:12.958254 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:12.958075 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="prom-label-proxy" containerID="cri-o://295ca3650f814f064f4aa76a19df95cca00ccae5b2ea6d6cb7f14f979bcb4333" gracePeriod=120 Jul 06 14:03:12.958254 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:12.958149 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy" containerID="cri-o://fe949c895700be13fa0dba57d4d74c6abe23b5fe78d6fca0e82ae5be07a47604" gracePeriod=120 Jul 06 14:03:12.958254 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:12.958250 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-web" containerID="cri-o://313d7b5cb87334599120590d1c548bc8488abcd7b7c990028af28f7760bd528a" gracePeriod=120 Jul 06 14:03:13.183351 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183319 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerID="295ca3650f814f064f4aa76a19df95cca00ccae5b2ea6d6cb7f14f979bcb4333" exitCode=0 Jul 06 14:03:13.183351 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183345 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerID="1086f00e1d68b097cc6f5338208ef7a05da507630e42b9a1af41c976783fc147" exitCode=0 Jul 06 14:03:13.183351 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183352 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerID="fe949c895700be13fa0dba57d4d74c6abe23b5fe78d6fca0e82ae5be07a47604" exitCode=0 Jul 06 14:03:13.183351 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183358 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerID="b0b02aa4d08f251fcf3174b5cb063c4c70165eb400a0e456de3ea8159b39f84c" exitCode=0 Jul 06 14:03:13.183351 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183363 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerID="e3c5588bc568d71c2a2fadb23094434edbc05b682d984602ddce30e669c40e97" exitCode=0 Jul 06 14:03:13.183713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183396 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"295ca3650f814f064f4aa76a19df95cca00ccae5b2ea6d6cb7f14f979bcb4333"} Jul 06 14:03:13.183713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183476 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"1086f00e1d68b097cc6f5338208ef7a05da507630e42b9a1af41c976783fc147"} Jul 06 14:03:13.183713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"fe949c895700be13fa0dba57d4d74c6abe23b5fe78d6fca0e82ae5be07a47604"} Jul 06 14:03:13.183713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"b0b02aa4d08f251fcf3174b5cb063c4c70165eb400a0e456de3ea8159b39f84c"} Jul 06 14:03:13.183713 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:13.183596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"e3c5588bc568d71c2a2fadb23094434edbc05b682d984602ddce30e669c40e97"} Jul 06 14:03:14.192716 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.192675 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerID="313d7b5cb87334599120590d1c548bc8488abcd7b7c990028af28f7760bd528a" exitCode=0 Jul 06 14:03:14.193094 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.192734 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"313d7b5cb87334599120590d1c548bc8488abcd7b7c990028af28f7760bd528a"} Jul 06 14:03:14.302849 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.302821 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:14.367785 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367698 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-trusted-ca-bundle\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.367785 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367746 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-web-config\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.367785 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367764 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-volume\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367788 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-main-db\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367810 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-web\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367838 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367853 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-out\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367883 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-metrics-client-ca\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367922 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.367975 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-tls-assets\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.368002 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.368031 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-cluster-tls-config\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368517 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.368094 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4mnt\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-kube-api-access-x4mnt\") pod \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\" (UID: \"3bff8145-d4d5-441c-af7d-7bd8e3309f7c\") " Jul 06 14:03:14.368517 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.368170 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:03:14.368517 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.368200 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:14.368517 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.368434 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-main-db\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.368517 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.368454 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.369415 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.369386 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:14.371382 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.371351 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:14.371742 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.371640 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:14.372055 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.372031 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-volume" (OuterVolumeSpecName: "config-volume") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:14.372604 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.372498 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:14.372604 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.372592 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:14.372792 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.372766 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-kube-api-access-x4mnt" (OuterVolumeSpecName: "kube-api-access-x4mnt") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "kube-api-access-x4mnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:03:14.372859 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.372839 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:03:14.373100 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.373078 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-out" (OuterVolumeSpecName: "config-out") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:03:14.376616 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.376584 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:14.383080 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.383058 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-web-config" (OuterVolumeSpecName: "web-config") pod "3bff8145-d4d5-441c-af7d-7bd8e3309f7c" (UID: "3bff8145-d4d5-441c-af7d-7bd8e3309f7c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:14.469434 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469380 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4mnt\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-kube-api-access-x4mnt\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469434 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469426 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-web-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469434 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469437 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-volume\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469434 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469447 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469457 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469465 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-config-out\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469474 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-metrics-client-ca\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469481 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-main-tls\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469489 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-tls-assets\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469497 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:14.469700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:14.469506 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bff8145-d4d5-441c-af7d-7bd8e3309f7c-cluster-tls-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:15.200091 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.200058 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bff8145-d4d5-441c-af7d-7bd8e3309f7c","Type":"ContainerDied","Data":"451ef1c6b3b065eb46c7824aa7c2c75a5ea4e927003822735b7b27eb26fc6419"} Jul 06 14:03:15.200493 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.200115 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.200493 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.200134 2562 scope.go:117] "RemoveContainer" containerID="295ca3650f814f064f4aa76a19df95cca00ccae5b2ea6d6cb7f14f979bcb4333" Jul 06 14:03:15.209139 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.208802 2562 scope.go:117] "RemoveContainer" containerID="1086f00e1d68b097cc6f5338208ef7a05da507630e42b9a1af41c976783fc147" Jul 06 14:03:15.216757 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.216735 2562 scope.go:117] "RemoveContainer" containerID="fe949c895700be13fa0dba57d4d74c6abe23b5fe78d6fca0e82ae5be07a47604" Jul 06 14:03:15.223391 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.223373 2562 scope.go:117] "RemoveContainer" containerID="313d7b5cb87334599120590d1c548bc8488abcd7b7c990028af28f7760bd528a" Jul 06 14:03:15.227965 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.227891 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:03:15.234068 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.234048 2562 scope.go:117] "RemoveContainer" containerID="b0b02aa4d08f251fcf3174b5cb063c4c70165eb400a0e456de3ea8159b39f84c" Jul 06 14:03:15.234525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.234502 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:03:15.241239 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.241220 2562 scope.go:117] "RemoveContainer" containerID="e3c5588bc568d71c2a2fadb23094434edbc05b682d984602ddce30e669c40e97" Jul 06 14:03:15.248298 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.248282 2562 scope.go:117] "RemoveContainer" containerID="7613e0076de8dc9cee12dc87f5acc66ccafab755b0e22cd9f31f37ea7e2aeebe" Jul 06 14:03:15.260031 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260007 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:03:15.260366 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260352 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="alertmanager" Jul 06 14:03:15.260411 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260367 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="alertmanager" Jul 06 14:03:15.260411 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260385 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="init-config-reloader" Jul 06 14:03:15.260411 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260395 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="init-config-reloader" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260413 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-metric" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260421 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-metric" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260433 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="348fadc9-b714-4bdc-983a-bfe26b9ebfac" containerName="console" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260440 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="348fadc9-b714-4bdc-983a-bfe26b9ebfac" containerName="console" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260454 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260463 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260474 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-web" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260482 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-web" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260493 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="prom-label-proxy" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260502 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="prom-label-proxy" Jul 06 14:03:15.260512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260516 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" containerName="console" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260525 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" containerName="console" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260537 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="config-reloader" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260543 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="config-reloader" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260603 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-web" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260611 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="alertmanager" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260618 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="348fadc9-b714-4bdc-983a-bfe26b9ebfac" containerName="console" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260623 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="prom-label-proxy" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260629 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy-metric" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260634 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ce89a4-f3cf-44f4-bc5f-e4ed6f48eb81" containerName="console" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260641 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="kube-rbac-proxy" Jul 06 14:03:15.260850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.260649 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" containerName="config-reloader" Jul 06 14:03:15.266033 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.266014 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.269000 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.268963 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 14:03:15.269118 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 14:03:15.269118 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269061 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 14:03:15.269544 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269526 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 14:03:15.269633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269542 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 14:03:15.269633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269566 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 14:03:15.269633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269566 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t45qr\"" Jul 06 14:03:15.269775 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269526 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 14:03:15.269962 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.269950 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 14:03:15.274050 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.274029 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 14:03:15.278186 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.278166 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:03:15.376779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.376740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.376779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.376778 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.376795 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.376853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrdng\" (UniqueName: \"kubernetes.io/projected/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-kube-api-access-nrdng\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.376937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.376971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-config-volume\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.377021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.377053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377358 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.377100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377358 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.377122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-web-config\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377358 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.377147 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377358 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.377184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.377358 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.377223 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-config-out\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.387536 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.387509 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bff8145-d4d5-441c-af7d-7bd8e3309f7c" path="/var/lib/kubelet/pods/3bff8145-d4d5-441c-af7d-7bd8e3309f7c/volumes" Jul 06 14:03:15.478151 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478151 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478151 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrdng\" (UniqueName: \"kubernetes.io/projected/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-kube-api-access-nrdng\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478270 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-config-volume\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-web-config\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478821 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.478821 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.478585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-config-out\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.479602 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.479248 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.479602 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.479301 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.481378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.481349 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.481489 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.481477 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.481548 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.481477 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-config-volume\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.481606 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.481585 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.481665 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.481632 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-config-out\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.481779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.481758 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.481930 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.481890 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.482269 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.482251 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.483612 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.483596 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-web-config\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.487596 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.487572 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrdng\" (UniqueName: \"kubernetes.io/projected/d548fc36-d7cd-4dd7-9b4d-5f48a58624da-kube-api-access-nrdng\") pod \"alertmanager-main-0\" (UID: \"d548fc36-d7cd-4dd7-9b4d-5f48a58624da\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.576727 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.576688 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 14:03:15.714321 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:15.714289 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 14:03:15.716416 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:03:15.716385 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd548fc36_d7cd_4dd7_9b4d_5f48a58624da.slice/crio-0708088fd53f4e5686b2d3b95f8d9d3acf59e6dc676b392fbaf671a45e9b9170 WatchSource:0}: Error finding container 0708088fd53f4e5686b2d3b95f8d9d3acf59e6dc676b392fbaf671a45e9b9170: Status 404 returned error can't find the container with id 0708088fd53f4e5686b2d3b95f8d9d3acf59e6dc676b392fbaf671a45e9b9170 Jul 06 14:03:16.205547 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:16.205512 2562 generic.go:356] "Generic (PLEG): container finished" podID="d548fc36-d7cd-4dd7-9b4d-5f48a58624da" containerID="c88232d9d2fc4b874b0179a47736c0b0ef10e5befbb36dd5307e052ba2af700e" exitCode=0 Jul 06 14:03:16.206007 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:16.205605 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerDied","Data":"c88232d9d2fc4b874b0179a47736c0b0ef10e5befbb36dd5307e052ba2af700e"} Jul 06 14:03:16.206007 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:16.205691 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerStarted","Data":"0708088fd53f4e5686b2d3b95f8d9d3acf59e6dc676b392fbaf671a45e9b9170"} Jul 06 14:03:17.010825 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.010782 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl"] Jul 06 14:03:17.013469 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.013450 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.018804 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.018762 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 14:03:17.018804 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.018762 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 14:03:17.019036 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.018916 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 14:03:17.019150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.019128 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 14:03:17.019212 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.019155 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rkwwn\"" Jul 06 14:03:17.019547 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.019529 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 14:03:17.025606 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.025584 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 14:03:17.031337 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.031305 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl"] Jul 06 14:03:17.093200 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlfjn\" (UniqueName: \"kubernetes.io/projected/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-kube-api-access-xlfjn\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.093373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093217 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-telemeter-client-tls\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.093373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.093373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-secret-telemeter-client\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.093373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.093529 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-federate-client-tls\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.093529 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-metrics-client-ca\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.093529 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.093490 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-serving-certs-ca-bundle\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194136 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-metrics-client-ca\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194308 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-serving-certs-ca-bundle\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194308 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlfjn\" (UniqueName: \"kubernetes.io/projected/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-kube-api-access-xlfjn\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194308 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-telemeter-client-tls\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194308 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194516 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-secret-telemeter-client\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194593 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.194653 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.194633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-federate-client-tls\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.195032 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.195002 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-metrics-client-ca\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.195621 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.195588 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-serving-certs-ca-bundle\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.196118 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.196095 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.197015 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.196969 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.197218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.197166 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-telemeter-client-tls\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.197333 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.197314 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-federate-client-tls\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.197431 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.197414 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-secret-telemeter-client\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.204618 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.204588 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlfjn\" (UniqueName: \"kubernetes.io/projected/e3acfa3e-43d9-4447-908b-0bc7f57c2e75-kube-api-access-xlfjn\") pod \"telemeter-client-5c44dd9c4c-7s9rl\" (UID: \"e3acfa3e-43d9-4447-908b-0bc7f57c2e75\") " pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.215811 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.215776 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerStarted","Data":"f41a56bed9ebc98f2f7330f4c1b35025a0856fe1ec509a12c0dc4a2581d9234e"} Jul 06 14:03:17.216170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.215848 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerStarted","Data":"a6e7e59c1155bc7b7a6e17e3b0ae87cb7a5567019edc34bee8eaab784e675596"} Jul 06 14:03:17.216170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.215875 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerStarted","Data":"6215f8e5fe35fe38ff77c5399b3c37c2e1ad9b52325acd4988ad6ddb52eade60"} Jul 06 14:03:17.216170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.215922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerStarted","Data":"e125a5313815fedb027f5e9bafc62b6248ba2200ca5809d5bd46090c7ed036b0"} Jul 06 14:03:17.216170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.215947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerStarted","Data":"220c6530e1e14bc3134a274cd8a7ef86de4cf132c0c266a45998934aabb54a14"} Jul 06 14:03:17.216170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.215971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d548fc36-d7cd-4dd7-9b4d-5f48a58624da","Type":"ContainerStarted","Data":"1d21b346870b563718682d75412c8326d0e950a6c4e947a35f12483e83fb76c7"} Jul 06 14:03:17.244362 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.244296 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.227581022 podStartE2EDuration="2.244273497s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.692475ms" isStatefulPod=false podCreationTimestamp="2026-07-06 14:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:03:17.243314723 +0000 UTC m=+148.432363770" watchObservedRunningTime="2026-07-06 14:03:17.244273497 +0000 UTC m=+148.433322544" Jul 06 14:03:17.324004 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.323910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" Jul 06 14:03:17.337204 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.337166 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:03:17.337581 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.337538 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="prometheus" containerID="cri-o://de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" gracePeriod=600 Jul 06 14:03:17.337994 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.337952 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" gracePeriod=600 Jul 06 14:03:17.338109 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.338086 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy" containerID="cri-o://2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" gracePeriod=600 Jul 06 14:03:17.338201 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.338180 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-web" containerID="cri-o://3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" gracePeriod=600 Jul 06 14:03:17.338286 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.338266 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="thanos-sidecar" containerID="cri-o://1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" gracePeriod=600 Jul 06 14:03:17.338374 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.338352 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="config-reloader" containerID="cri-o://1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" gracePeriod=600 Jul 06 14:03:17.508633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.508600 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl"] Jul 06 14:03:17.510364 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:03:17.510333 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3acfa3e_43d9_4447_908b_0bc7f57c2e75.slice/crio-e3c1a553d00c0ec67846e44a0acbe53af6d2224777fe0672e1dc44eddc5d9af3 WatchSource:0}: Error finding container e3c1a553d00c0ec67846e44a0acbe53af6d2224777fe0672e1dc44eddc5d9af3: Status 404 returned error can't find the container with id e3c1a553d00c0ec67846e44a0acbe53af6d2224777fe0672e1dc44eddc5d9af3 Jul 06 14:03:17.631263 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.631237 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:17.699789 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.699759 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-grpc-tls\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.699789 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.699791 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-metrics-client-certs\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.699848 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-config-out\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.699873 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-metrics-client-ca\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.699931 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-web-config\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.699958 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-thanos-prometheus-http-client-file\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.699988 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700012 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-trusted-ca-bundle\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700036 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-kube-rbac-proxy\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700060 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-config\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700081 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-serving-certs-ca-bundle\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700116 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700111 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-tls\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700140 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t8fq\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-kube-api-access-9t8fq\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-0\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700221 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-tls-assets\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700260 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700287 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-2\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700342 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-kubelet-serving-ca-bundle\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700370 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-1\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700412 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-db\") pod \"318f09bd-de33-4503-ad06-e7d31b7b8865\" (UID: \"318f09bd-de33-4503-ad06-e7d31b7b8865\") " Jul 06 14:03:17.700559 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700494 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:17.700985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.700752 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.701093 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.701063 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:17.701466 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.701441 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:17.701577 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.701553 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:17.702975 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.702945 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:17.703326 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.703287 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:17.703671 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.703651 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:17.704168 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.704142 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:03:17.704616 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.704579 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.705678 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.705649 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-kube-api-access-9t8fq" (OuterVolumeSpecName: "kube-api-access-9t8fq") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "kube-api-access-9t8fq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:03:17.705847 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.705748 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-config-out" (OuterVolumeSpecName: "config-out") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:03:17.705847 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.705819 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.706019 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.705980 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.706088 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.706055 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.706509 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.706363 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:03:17.706509 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.706413 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.706509 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.706483 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.706726 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.706513 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-config" (OuterVolumeSpecName: "config") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.707073 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.707051 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.718849 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.718812 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-web-config" (OuterVolumeSpecName: "web-config") pod "318f09bd-de33-4503-ad06-e7d31b7b8865" (UID: "318f09bd-de33-4503-ad06-e7d31b7b8865"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:17.801995 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.801953 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-config-out\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.801995 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.801986 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-metrics-client-ca\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.801995 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.801999 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-web-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802011 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802024 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802039 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-kube-rbac-proxy\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802050 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802063 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802074 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802088 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t8fq\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-kube-api-access-9t8fq\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802099 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802110 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/318f09bd-de33-4503-ad06-e7d31b7b8865-tls-assets\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802122 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802134 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802145 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802157 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802170 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/318f09bd-de33-4503-ad06-e7d31b7b8865-prometheus-k8s-db\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802182 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-grpc-tls\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:17.802270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:17.802196 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/318f09bd-de33-4503-ad06-e7d31b7b8865-secret-metrics-client-certs\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:18.223585 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.223545 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" event={"ID":"e3acfa3e-43d9-4447-908b-0bc7f57c2e75","Type":"ContainerStarted","Data":"e3c1a553d00c0ec67846e44a0acbe53af6d2224777fe0672e1dc44eddc5d9af3"} Jul 06 14:03:18.227459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227428 2562 generic.go:356] "Generic (PLEG): container finished" podID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" exitCode=0 Jul 06 14:03:18.227459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227456 2562 generic.go:356] "Generic (PLEG): container finished" podID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" exitCode=0 Jul 06 14:03:18.227661 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227469 2562 generic.go:356] "Generic (PLEG): container finished" podID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" exitCode=0 Jul 06 14:03:18.227661 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227478 2562 generic.go:356] "Generic (PLEG): container finished" podID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" exitCode=0 Jul 06 14:03:18.227661 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227487 2562 generic.go:356] "Generic (PLEG): container finished" podID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" exitCode=0 Jul 06 14:03:18.227661 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227495 2562 generic.go:356] "Generic (PLEG): container finished" podID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" exitCode=0 Jul 06 14:03:18.227661 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227510 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.227661 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227508 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} Jul 06 14:03:18.228143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.228111 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} Jul 06 14:03:18.228244 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.228225 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} Jul 06 14:03:18.233931 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.227595 2562 scope.go:117] "RemoveContainer" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.233931 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.233845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} Jul 06 14:03:18.234119 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.233980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} Jul 06 14:03:18.234119 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.234037 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} Jul 06 14:03:18.234119 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.234073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"318f09bd-de33-4503-ad06-e7d31b7b8865","Type":"ContainerDied","Data":"b9ec1fda8572b60879c552e103dd8d0ab390eeaa0e09faabbb78097de096f806"} Jul 06 14:03:18.244391 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.244364 2562 scope.go:117] "RemoveContainer" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.253848 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.253820 2562 scope.go:117] "RemoveContainer" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.262796 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.262763 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:03:18.265621 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.265517 2562 scope.go:117] "RemoveContainer" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.269157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.269134 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:03:18.275043 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.275024 2562 scope.go:117] "RemoveContainer" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.282336 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.282315 2562 scope.go:117] "RemoveContainer" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.291401 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.291376 2562 scope.go:117] "RemoveContainer" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.298735 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.298716 2562 scope.go:117] "RemoveContainer" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.299008 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:18.298991 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": container with ID starting with e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c not found: ID does not exist" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.299095 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.299016 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} err="failed to get container status \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": rpc error: code = NotFound desc = could not find container \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": container with ID starting with e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c not found: ID does not exist" Jul 06 14:03:18.299095 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.299036 2562 scope.go:117] "RemoveContainer" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.299262 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:18.299243 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": container with ID starting with 2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349 not found: ID does not exist" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.299326 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.299272 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} err="failed to get container status \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": rpc error: code = NotFound desc = could not find container \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": container with ID starting with 2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349 not found: ID does not exist" Jul 06 14:03:18.299326 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.299297 2562 scope.go:117] "RemoveContainer" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.299556 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:18.299536 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": container with ID starting with 3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485 not found: ID does not exist" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.299628 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.299563 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} err="failed to get container status \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": rpc error: code = NotFound desc = could not find container \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": container with ID starting with 3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485 not found: ID does not exist" Jul 06 14:03:18.299628 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.299583 2562 scope.go:117] "RemoveContainer" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.300480 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:18.300349 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": container with ID starting with 1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418 not found: ID does not exist" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.300480 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.300389 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} err="failed to get container status \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": rpc error: code = NotFound desc = could not find container \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": container with ID starting with 1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418 not found: ID does not exist" Jul 06 14:03:18.300480 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.300408 2562 scope.go:117] "RemoveContainer" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.300781 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:18.300732 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": container with ID starting with 1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254 not found: ID does not exist" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.300781 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.300760 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} err="failed to get container status \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": rpc error: code = NotFound desc = could not find container \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": container with ID starting with 1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254 not found: ID does not exist" Jul 06 14:03:18.300781 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.300781 2562 scope.go:117] "RemoveContainer" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.301243 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:18.301218 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": container with ID starting with de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862 not found: ID does not exist" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.301348 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.301249 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} err="failed to get container status \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": rpc error: code = NotFound desc = could not find container \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": container with ID starting with de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862 not found: ID does not exist" Jul 06 14:03:18.301348 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.301269 2562 scope.go:117] "RemoveContainer" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.301530 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:18.301512 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": container with ID starting with f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1 not found: ID does not exist" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.301530 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.301533 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1"} err="failed to get container status \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": rpc error: code = NotFound desc = could not find container \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": container with ID starting with f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1 not found: ID does not exist" Jul 06 14:03:18.301667 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.301545 2562 scope.go:117] "RemoveContainer" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.301797 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.301769 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} err="failed to get container status \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": rpc error: code = NotFound desc = could not find container \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": container with ID starting with e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c not found: ID does not exist" Jul 06 14:03:18.301933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.301797 2562 scope.go:117] "RemoveContainer" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.302110 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302080 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} err="failed to get container status \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": rpc error: code = NotFound desc = could not find container \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": container with ID starting with 2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349 not found: ID does not exist" Jul 06 14:03:18.302180 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302113 2562 scope.go:117] "RemoveContainer" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.302363 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302339 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} err="failed to get container status \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": rpc error: code = NotFound desc = could not find container \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": container with ID starting with 3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485 not found: ID does not exist" Jul 06 14:03:18.302363 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302362 2562 scope.go:117] "RemoveContainer" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.302658 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302626 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} err="failed to get container status \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": rpc error: code = NotFound desc = could not find container \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": container with ID starting with 1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418 not found: ID does not exist" Jul 06 14:03:18.302658 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302653 2562 scope.go:117] "RemoveContainer" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.302874 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302856 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:03:18.302874 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302861 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} err="failed to get container status \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": rpc error: code = NotFound desc = could not find container \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": container with ID starting with 1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254 not found: ID does not exist" Jul 06 14:03:18.303079 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.302884 2562 scope.go:117] "RemoveContainer" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.303139 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303121 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} err="failed to get container status \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": rpc error: code = NotFound desc = could not find container \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": container with ID starting with de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862 not found: ID does not exist" Jul 06 14:03:18.303186 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303141 2562 scope.go:117] "RemoveContainer" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.303342 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303324 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1"} err="failed to get container status \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": rpc error: code = NotFound desc = could not find container \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": container with ID starting with f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1 not found: ID does not exist" Jul 06 14:03:18.303402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303344 2562 scope.go:117] "RemoveContainer" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.303402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303329 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="config-reloader" Jul 06 14:03:18.303402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303366 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="config-reloader" Jul 06 14:03:18.303402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303384 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy" Jul 06 14:03:18.303402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303392 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303406 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="prometheus" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303415 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="prometheus" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303431 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-thanos" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303436 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-thanos" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303455 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="init-config-reloader" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303464 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="init-config-reloader" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303481 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="thanos-sidecar" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303489 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="thanos-sidecar" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303506 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-web" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303512 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-web" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303591 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-web" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303599 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="prometheus" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303606 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="config-reloader" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303612 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy" Jul 06 14:03:18.303620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303602 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} err="failed to get container status \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": rpc error: code = NotFound desc = could not find container \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": container with ID starting with e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c not found: ID does not exist" Jul 06 14:03:18.304169 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303633 2562 scope.go:117] "RemoveContainer" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.304169 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303617 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="thanos-sidecar" Jul 06 14:03:18.304169 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303704 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" containerName="kube-rbac-proxy-thanos" Jul 06 14:03:18.304169 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303949 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} err="failed to get container status \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": rpc error: code = NotFound desc = could not find container \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": container with ID starting with 2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349 not found: ID does not exist" Jul 06 14:03:18.304169 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.303971 2562 scope.go:117] "RemoveContainer" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.304349 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.304189 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} err="failed to get container status \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": rpc error: code = NotFound desc = could not find container \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": container with ID starting with 3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485 not found: ID does not exist" Jul 06 14:03:18.304349 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.304218 2562 scope.go:117] "RemoveContainer" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.304478 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.304462 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} err="failed to get container status \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": rpc error: code = NotFound desc = could not find container \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": container with ID starting with 1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418 not found: ID does not exist" Jul 06 14:03:18.304521 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.304479 2562 scope.go:117] "RemoveContainer" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.305038 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.304830 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} err="failed to get container status \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": rpc error: code = NotFound desc = could not find container \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": container with ID starting with 1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254 not found: ID does not exist" Jul 06 14:03:18.305134 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.305040 2562 scope.go:117] "RemoveContainer" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.305416 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.305389 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} err="failed to get container status \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": rpc error: code = NotFound desc = could not find container \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": container with ID starting with de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862 not found: ID does not exist" Jul 06 14:03:18.305487 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.305418 2562 scope.go:117] "RemoveContainer" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.305751 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.305711 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1"} err="failed to get container status \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": rpc error: code = NotFound desc = could not find container \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": container with ID starting with f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1 not found: ID does not exist" Jul 06 14:03:18.305849 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.305752 2562 scope.go:117] "RemoveContainer" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.306552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.306527 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} err="failed to get container status \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": rpc error: code = NotFound desc = could not find container \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": container with ID starting with e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c not found: ID does not exist" Jul 06 14:03:18.306552 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.306551 2562 scope.go:117] "RemoveContainer" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.306757 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.306742 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} err="failed to get container status \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": rpc error: code = NotFound desc = could not find container \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": container with ID starting with 2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349 not found: ID does not exist" Jul 06 14:03:18.306798 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.306757 2562 scope.go:117] "RemoveContainer" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.307011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.306979 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} err="failed to get container status \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": rpc error: code = NotFound desc = could not find container \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": container with ID starting with 3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485 not found: ID does not exist" Jul 06 14:03:18.307011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307001 2562 scope.go:117] "RemoveContainer" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.307260 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307240 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} err="failed to get container status \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": rpc error: code = NotFound desc = could not find container \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": container with ID starting with 1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418 not found: ID does not exist" Jul 06 14:03:18.307260 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307260 2562 scope.go:117] "RemoveContainer" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.307495 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307478 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} err="failed to get container status \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": rpc error: code = NotFound desc = could not find container \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": container with ID starting with 1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254 not found: ID does not exist" Jul 06 14:03:18.307558 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307496 2562 scope.go:117] "RemoveContainer" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.307730 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307714 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} err="failed to get container status \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": rpc error: code = NotFound desc = could not find container \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": container with ID starting with de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862 not found: ID does not exist" Jul 06 14:03:18.307780 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307730 2562 scope.go:117] "RemoveContainer" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.307984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307955 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1"} err="failed to get container status \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": rpc error: code = NotFound desc = could not find container \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": container with ID starting with f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1 not found: ID does not exist" Jul 06 14:03:18.307984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.307987 2562 scope.go:117] "RemoveContainer" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.308238 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.308220 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} err="failed to get container status \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": rpc error: code = NotFound desc = could not find container \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": container with ID starting with e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c not found: ID does not exist" Jul 06 14:03:18.308306 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.308239 2562 scope.go:117] "RemoveContainer" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.308447 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.308430 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} err="failed to get container status \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": rpc error: code = NotFound desc = could not find container \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": container with ID starting with 2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349 not found: ID does not exist" Jul 06 14:03:18.308498 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.308448 2562 scope.go:117] "RemoveContainer" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.308546 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.308528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.308679 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.308656 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} err="failed to get container status \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": rpc error: code = NotFound desc = could not find container \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": container with ID starting with 3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485 not found: ID does not exist" Jul 06 14:03:18.308679 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.308677 2562 scope.go:117] "RemoveContainer" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.309186 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309163 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} err="failed to get container status \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": rpc error: code = NotFound desc = could not find container \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": container with ID starting with 1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418 not found: ID does not exist" Jul 06 14:03:18.309186 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309186 2562 scope.go:117] "RemoveContainer" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.309439 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309420 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} err="failed to get container status \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": rpc error: code = NotFound desc = could not find container \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": container with ID starting with 1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254 not found: ID does not exist" Jul 06 14:03:18.309490 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309440 2562 scope.go:117] "RemoveContainer" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.309671 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309654 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} err="failed to get container status \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": rpc error: code = NotFound desc = could not find container \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": container with ID starting with de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862 not found: ID does not exist" Jul 06 14:03:18.309717 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309672 2562 scope.go:117] "RemoveContainer" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.309959 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309938 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1"} err="failed to get container status \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": rpc error: code = NotFound desc = could not find container \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": container with ID starting with f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1 not found: ID does not exist" Jul 06 14:03:18.309959 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.309959 2562 scope.go:117] "RemoveContainer" containerID="e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c" Jul 06 14:03:18.310197 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310179 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c"} err="failed to get container status \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": rpc error: code = NotFound desc = could not find container \"e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c\": container with ID starting with e14f978a70be0f6eb8e415c61513696fda96f2d5bcb167485ae2fb7923eea64c not found: ID does not exist" Jul 06 14:03:18.310275 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310199 2562 scope.go:117] "RemoveContainer" containerID="2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349" Jul 06 14:03:18.310435 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310418 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349"} err="failed to get container status \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": rpc error: code = NotFound desc = could not find container \"2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349\": container with ID starting with 2f125e29b6755a2dfa4e3b08b494d1efe1ecfdf703497832f9cfca4aab4e6349 not found: ID does not exist" Jul 06 14:03:18.310482 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310436 2562 scope.go:117] "RemoveContainer" containerID="3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485" Jul 06 14:03:18.310653 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310638 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485"} err="failed to get container status \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": rpc error: code = NotFound desc = could not find container \"3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485\": container with ID starting with 3ebd25e4fbe7f7507af52686e3988bc0fa49998658c0d6e7e550c9d6a6f4a485 not found: ID does not exist" Jul 06 14:03:18.310699 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310653 2562 scope.go:117] "RemoveContainer" containerID="1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418" Jul 06 14:03:18.310865 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310845 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418"} err="failed to get container status \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": rpc error: code = NotFound desc = could not find container \"1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418\": container with ID starting with 1691893fe699a00d07a0ecc32e89cd53a37687d3adc22461a5ff83bf22b9b418 not found: ID does not exist" Jul 06 14:03:18.310962 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.310866 2562 scope.go:117] "RemoveContainer" containerID="1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254" Jul 06 14:03:18.311303 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311232 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254"} err="failed to get container status \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": rpc error: code = NotFound desc = could not find container \"1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254\": container with ID starting with 1a1d1e7ce6a3a3f295efb29e7d125750e3802d2f732d1dc9710b73ada6c17254 not found: ID does not exist" Jul 06 14:03:18.311388 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311304 2562 scope.go:117] "RemoveContainer" containerID="de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862" Jul 06 14:03:18.311388 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311373 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 14:03:18.311494 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311377 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 14:03:18.311582 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311558 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862"} err="failed to get container status \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": rpc error: code = NotFound desc = could not find container \"de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862\": container with ID starting with de1339ca2aae550ec09d372b66f1bf10b243b652a99e2074befbb9300685f862 not found: ID does not exist" Jul 06 14:03:18.311645 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311583 2562 scope.go:117] "RemoveContainer" containerID="f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1" Jul 06 14:03:18.311824 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311795 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 14:03:18.311932 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311826 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 14:03:18.312004 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.311954 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 14:03:18.312063 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312037 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1"} err="failed to get container status \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": rpc error: code = NotFound desc = could not find container \"f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1\": container with ID starting with f59d68685a43ae79dfbb906e2f6463eb72572a887ae5232b406511a4377ee7a1 not found: ID does not exist" Jul 06 14:03:18.312135 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312091 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 14:03:18.312500 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312345 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a4kpp1o720qt5\"" Jul 06 14:03:18.312500 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312381 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-q2vdj\"" Jul 06 14:03:18.312500 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312389 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 14:03:18.312500 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312472 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 14:03:18.313317 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312652 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 14:03:18.313317 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.312834 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 14:03:18.314470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.314397 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 14:03:18.314639 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.314411 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 14:03:18.315985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.315962 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 14:03:18.317705 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.317689 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 14:03:18.350967 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.336289 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:03:18.407114 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407114 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407110 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407317 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407317 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407317 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407317 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407445 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-config\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407445 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407540 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407598 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407654 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407596 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407654 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407621 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407728 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407761 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407733 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407800 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407881 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407959 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.407996 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407968 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.408032 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.407991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnpcw\" (UniqueName: \"kubernetes.io/projected/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-kube-api-access-cnpcw\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.408065 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.408035 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.508981 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.508947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-config\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.508996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509237 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509303 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnpcw\" (UniqueName: \"kubernetes.io/projected/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-kube-api-access-cnpcw\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509394 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509448 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509994 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509486 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.509994 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.509524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.510217 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.510189 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.510469 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.510442 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.510682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.510662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.512416 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.512388 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.512923 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.512880 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-config\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.513007 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.512976 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.513007 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.512985 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.513198 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.513173 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-config-out\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.514011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.513984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.514011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.513991 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.514340 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.514317 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.514970 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.514944 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.515142 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.515113 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.515369 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.515231 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-web-config\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.515893 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.515696 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.516008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.515956 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.516847 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.516827 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.516976 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.516884 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.517302 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.517280 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.520632 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.520609 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnpcw\" (UniqueName: \"kubernetes.io/projected/f8a53b7e-1165-4dac-be6e-2923a7a57aa9-kube-api-access-cnpcw\") pod \"prometheus-k8s-0\" (UID: \"f8a53b7e-1165-4dac-be6e-2923a7a57aa9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.622402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.622370 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:18.776399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:18.776366 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 14:03:18.779869 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:03:18.779840 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a53b7e_1165_4dac_be6e_2923a7a57aa9.slice/crio-ab1daa7748dfe4b898407c9c924a0ee574c1f146c1f0a82cf4566dffe4414736 WatchSource:0}: Error finding container ab1daa7748dfe4b898407c9c924a0ee574c1f146c1f0a82cf4566dffe4414736: Status 404 returned error can't find the container with id ab1daa7748dfe4b898407c9c924a0ee574c1f146c1f0a82cf4566dffe4414736 Jul 06 14:03:19.234646 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:19.234560 2562 generic.go:356] "Generic (PLEG): container finished" podID="f8a53b7e-1165-4dac-be6e-2923a7a57aa9" containerID="4129219b62f25a503a3c7d09339db0ba9c8e94dbd9353acf805a9ee9e2ad80cb" exitCode=0 Jul 06 14:03:19.235143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:19.234651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerDied","Data":"4129219b62f25a503a3c7d09339db0ba9c8e94dbd9353acf805a9ee9e2ad80cb"} Jul 06 14:03:19.235143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:19.234741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerStarted","Data":"ab1daa7748dfe4b898407c9c924a0ee574c1f146c1f0a82cf4566dffe4414736"} Jul 06 14:03:19.390212 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:19.390177 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="318f09bd-de33-4503-ad06-e7d31b7b8865" path="/var/lib/kubelet/pods/318f09bd-de33-4503-ad06-e7d31b7b8865/volumes" Jul 06 14:03:20.243479 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.243382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerStarted","Data":"fc6b08e47396b7be229615c8b22c117d10f2fadfffa5ab626c902535f7ca2523"} Jul 06 14:03:20.243922 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.243473 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerStarted","Data":"e689dce3009d2daf24784ce95499a7e3b90fe20b829c7140c2806c8b2590ba03"} Jul 06 14:03:20.243922 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.243521 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerStarted","Data":"99987c1c2cd667932fb6ed74462c1131e22cd8ad15128cf2f0cf5e711a070577"} Jul 06 14:03:20.243922 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.243564 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerStarted","Data":"26033d635a97936b8e263a1806532db8238de4101db72e8c1e405952d0d1008e"} Jul 06 14:03:20.243922 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.243597 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerStarted","Data":"378e92e0e6da60b9ec6d377cc94835a111a5fbf848b56ebcb59f71afefe57d47"} Jul 06 14:03:20.243922 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.243622 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f8a53b7e-1165-4dac-be6e-2923a7a57aa9","Type":"ContainerStarted","Data":"4b5c4a97b7d50fdd00c6e365bd631a214a86bcbe24bef6541598941d5c6d2ee3"} Jul 06 14:03:20.245525 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.245502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" event={"ID":"e3acfa3e-43d9-4447-908b-0bc7f57c2e75","Type":"ContainerStarted","Data":"21b73fb27d3d73f476f315eae0297d5f95c82719a8b3c503fa84fd35c797a8d6"} Jul 06 14:03:20.245633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.245552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" event={"ID":"e3acfa3e-43d9-4447-908b-0bc7f57c2e75","Type":"ContainerStarted","Data":"f9c8b39f7ae0ab4abb7a1ea82a8a91aebe64f9bb9fd51224cd0f5fd49cd34fa3"} Jul 06 14:03:20.245633 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.245576 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" event={"ID":"e3acfa3e-43d9-4447-908b-0bc7f57c2e75","Type":"ContainerStarted","Data":"5d23d4309d2680949cdcd53486145b60ac46a32d6aa443c368f0ed089a1fe0ee"} Jul 06 14:03:20.277028 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.276975 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.256595238 podStartE2EDuration="2.27695891s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.363672ms" isStatefulPod=false podCreationTimestamp="2026-07-06 14:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:03:20.274304756 +0000 UTC m=+151.463353815" watchObservedRunningTime="2026-07-06 14:03:20.27695891 +0000 UTC m=+151.466007956" Jul 06 14:03:20.299691 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:20.299638 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5c44dd9c4c-7s9rl" podStartSLOduration=2.117800901 podStartE2EDuration="4.299620578s" totalImagesPullingTime="2.181819677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:03:20.298474744 +0000 UTC m=+151.487523846" watchObservedRunningTime="2026-07-06 14:03:20.299620578 +0000 UTC m=+151.488669625" Jul 06 14:03:21.019652 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.019614 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-685649778d-46dvx"] Jul 06 14:03:21.022809 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.022784 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.035272 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.035244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-685649778d-46dvx"] Jul 06 14:03:21.136912 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.136822 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-serving-cert\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.137149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.136887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-trusted-ca-bundle\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.137149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.136983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-console-config\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.137149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.137039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-service-ca\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.137149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.137093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-oauth-config\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.137374 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.137183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-oauth-serving-cert\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.137374 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.137199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzw5h\" (UniqueName: \"kubernetes.io/projected/ac143289-c8b8-4721-9079-665ca189b950-kube-api-access-vzw5h\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.238562 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.238518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-oauth-serving-cert\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.238562 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.238558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzw5h\" (UniqueName: \"kubernetes.io/projected/ac143289-c8b8-4721-9079-665ca189b950-kube-api-access-vzw5h\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.238811 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.238664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-serving-cert\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.238966 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.238933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-trusted-ca-bundle\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.239129 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.239023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-console-config\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.239129 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.239061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-service-ca\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.239129 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.239087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-oauth-config\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.239733 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.239708 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-trusted-ca-bundle\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.239849 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.239705 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-service-ca\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.239849 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.239758 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-console-config\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.239980 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.239961 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-oauth-serving-cert\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.242805 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.241725 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-serving-cert\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.243154 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.243136 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-oauth-config\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.250326 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.250269 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzw5h\" (UniqueName: \"kubernetes.io/projected/ac143289-c8b8-4721-9079-665ca189b950-kube-api-access-vzw5h\") pod \"console-685649778d-46dvx\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.336146 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.336059 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:21.491080 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:21.491021 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-685649778d-46dvx"] Jul 06 14:03:21.495669 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:03:21.495641 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac143289_c8b8_4721_9079_665ca189b950.slice/crio-778595c918f933cccea8f09fe1fd23b7ec8cb4bff610ddbd83361f8a80c74e5b WatchSource:0}: Error finding container 778595c918f933cccea8f09fe1fd23b7ec8cb4bff610ddbd83361f8a80c74e5b: Status 404 returned error can't find the container with id 778595c918f933cccea8f09fe1fd23b7ec8cb4bff610ddbd83361f8a80c74e5b Jul 06 14:03:22.258278 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:22.258242 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685649778d-46dvx" event={"ID":"ac143289-c8b8-4721-9079-665ca189b950","Type":"ContainerStarted","Data":"4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68"} Jul 06 14:03:22.258640 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:22.258298 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685649778d-46dvx" event={"ID":"ac143289-c8b8-4721-9079-665ca189b950","Type":"ContainerStarted","Data":"778595c918f933cccea8f09fe1fd23b7ec8cb4bff610ddbd83361f8a80c74e5b"} Jul 06 14:03:22.282664 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:22.282613 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-685649778d-46dvx" podStartSLOduration=1.282595806 podStartE2EDuration="1.282595806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:03:22.280690118 +0000 UTC m=+153.469739185" watchObservedRunningTime="2026-07-06 14:03:22.282595806 +0000 UTC m=+153.471644852" Jul 06 14:03:23.622677 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:23.622638 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:03:31.336142 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:31.336103 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:31.336616 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:31.336166 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:31.340695 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:31.340670 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:32.309197 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:32.309169 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:03:32.357557 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:32.357524 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796bc448d6-p68v5"] Jul 06 14:03:57.377864 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.377819 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-796bc448d6-p68v5" podUID="26e6c50c-c876-4b2d-b63a-41cde6fb678b" containerName="console" containerID="cri-o://ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a" gracePeriod=15 Jul 06 14:03:57.631395 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.631330 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:03:57.678220 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678188 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-trusted-ca-bundle\") pod \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " Jul 06 14:03:57.678220 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678227 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsk29\" (UniqueName: \"kubernetes.io/projected/26e6c50c-c876-4b2d-b63a-41cde6fb678b-kube-api-access-qsk29\") pod \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " Jul 06 14:03:57.678473 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678260 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-serving-cert\") pod \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " Jul 06 14:03:57.678473 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678280 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-oauth-config\") pod \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " Jul 06 14:03:57.678473 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678298 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-config\") pod \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " Jul 06 14:03:57.678473 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678328 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-service-ca\") pod \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " Jul 06 14:03:57.678473 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678390 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-oauth-serving-cert\") pod \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\" (UID: \"26e6c50c-c876-4b2d-b63a-41cde6fb678b\") " Jul 06 14:03:57.678722 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678691 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "26e6c50c-c876-4b2d-b63a-41cde6fb678b" (UID: "26e6c50c-c876-4b2d-b63a-41cde6fb678b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:57.678856 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678825 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-service-ca" (OuterVolumeSpecName: "service-ca") pod "26e6c50c-c876-4b2d-b63a-41cde6fb678b" (UID: "26e6c50c-c876-4b2d-b63a-41cde6fb678b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:57.678948 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.678876 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "26e6c50c-c876-4b2d-b63a-41cde6fb678b" (UID: "26e6c50c-c876-4b2d-b63a-41cde6fb678b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:57.679037 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.679016 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-config" (OuterVolumeSpecName: "console-config") pod "26e6c50c-c876-4b2d-b63a-41cde6fb678b" (UID: "26e6c50c-c876-4b2d-b63a-41cde6fb678b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:03:57.680628 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.680603 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "26e6c50c-c876-4b2d-b63a-41cde6fb678b" (UID: "26e6c50c-c876-4b2d-b63a-41cde6fb678b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:57.680628 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.680598 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e6c50c-c876-4b2d-b63a-41cde6fb678b-kube-api-access-qsk29" (OuterVolumeSpecName: "kube-api-access-qsk29") pod "26e6c50c-c876-4b2d-b63a-41cde6fb678b" (UID: "26e6c50c-c876-4b2d-b63a-41cde6fb678b"). InnerVolumeSpecName "kube-api-access-qsk29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:03:57.680792 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.680682 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "26e6c50c-c876-4b2d-b63a-41cde6fb678b" (UID: "26e6c50c-c876-4b2d-b63a-41cde6fb678b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:03:57.779962 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.779924 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-oauth-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:57.779962 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.779955 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-trusted-ca-bundle\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:57.779962 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.779966 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsk29\" (UniqueName: \"kubernetes.io/projected/26e6c50c-c876-4b2d-b63a-41cde6fb678b-kube-api-access-qsk29\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:57.779962 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.779974 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:57.780217 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.779983 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-oauth-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:57.780217 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.779991 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-console-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:57.780217 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:57.780014 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e6c50c-c876-4b2d-b63a-41cde6fb678b-service-ca\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:03:58.414366 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.414333 2562 generic.go:356] "Generic (PLEG): container finished" podID="26e6c50c-c876-4b2d-b63a-41cde6fb678b" containerID="ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a" exitCode=0 Jul 06 14:03:58.414794 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.414381 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-796bc448d6-p68v5" Jul 06 14:03:58.414794 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.414415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bc448d6-p68v5" event={"ID":"26e6c50c-c876-4b2d-b63a-41cde6fb678b","Type":"ContainerDied","Data":"ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a"} Jul 06 14:03:58.414794 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.414469 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-796bc448d6-p68v5" event={"ID":"26e6c50c-c876-4b2d-b63a-41cde6fb678b","Type":"ContainerDied","Data":"6bf99e1a8afc017a5f5407ae5da07aee3d1c2a2fea63879375c8f0096a90f4c5"} Jul 06 14:03:58.414794 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.414490 2562 scope.go:117] "RemoveContainer" containerID="ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a" Jul 06 14:03:58.423589 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.423574 2562 scope.go:117] "RemoveContainer" containerID="ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a" Jul 06 14:03:58.423862 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:03:58.423842 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a\": container with ID starting with ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a not found: ID does not exist" containerID="ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a" Jul 06 14:03:58.423941 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.423876 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a"} err="failed to get container status \"ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a\": rpc error: code = NotFound desc = could not find container \"ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a\": container with ID starting with ef5c75d637f7c75f458da38ef3a1c90bca05fd2b7a17cd8f1b919fa24037853a not found: ID does not exist" Jul 06 14:03:58.438426 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.438402 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-796bc448d6-p68v5"] Jul 06 14:03:58.446022 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:58.445997 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-796bc448d6-p68v5"] Jul 06 14:03:59.387985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:03:59.387952 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26e6c50c-c876-4b2d-b63a-41cde6fb678b" path="/var/lib/kubelet/pods/26e6c50c-c876-4b2d-b63a-41cde6fb678b/volumes" Jul 06 14:04:18.622968 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:18.622934 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:04:18.639430 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:18.639405 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:04:19.520249 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:19.520215 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 14:04:27.457858 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.457823 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-696bb467c6-59g9z"] Jul 06 14:04:27.458438 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.458206 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e6c50c-c876-4b2d-b63a-41cde6fb678b" containerName="console" Jul 06 14:04:27.458438 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.458218 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e6c50c-c876-4b2d-b63a-41cde6fb678b" containerName="console" Jul 06 14:04:27.458438 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.458269 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e6c50c-c876-4b2d-b63a-41cde6fb678b" containerName="console" Jul 06 14:04:27.462423 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.462403 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.476991 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.476964 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696bb467c6-59g9z"] Jul 06 14:04:27.546264 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.546232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-console-config\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.546264 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.546263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33fe9478-5f00-4490-ba78-e12e226c18ff-console-serving-cert\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.546485 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.546303 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-oauth-serving-cert\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.546485 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.546364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33fe9478-5f00-4490-ba78-e12e226c18ff-console-oauth-config\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.546485 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.546395 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j22m4\" (UniqueName: \"kubernetes.io/projected/33fe9478-5f00-4490-ba78-e12e226c18ff-kube-api-access-j22m4\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.546485 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.546431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-trusted-ca-bundle\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.546485 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.546449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-service-ca\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.647589 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.647548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-oauth-serving-cert\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.647787 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.647604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33fe9478-5f00-4490-ba78-e12e226c18ff-console-oauth-config\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.647787 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.647638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j22m4\" (UniqueName: \"kubernetes.io/projected/33fe9478-5f00-4490-ba78-e12e226c18ff-kube-api-access-j22m4\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.647787 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.647672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-trusted-ca-bundle\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.647787 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.647701 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-service-ca\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.647787 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.647734 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-console-config\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.647787 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.647761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33fe9478-5f00-4490-ba78-e12e226c18ff-console-serving-cert\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.648484 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.648456 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-oauth-serving-cert\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.648558 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.648516 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-console-config\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.648603 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.648586 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-trusted-ca-bundle\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.648639 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.648595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33fe9478-5f00-4490-ba78-e12e226c18ff-service-ca\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.650121 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.650101 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33fe9478-5f00-4490-ba78-e12e226c18ff-console-serving-cert\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.650209 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.650141 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33fe9478-5f00-4490-ba78-e12e226c18ff-console-oauth-config\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.657950 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.657916 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j22m4\" (UniqueName: \"kubernetes.io/projected/33fe9478-5f00-4490-ba78-e12e226c18ff-kube-api-access-j22m4\") pod \"console-696bb467c6-59g9z\" (UID: \"33fe9478-5f00-4490-ba78-e12e226c18ff\") " pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.771111 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.771073 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:27.909995 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:27.909964 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-696bb467c6-59g9z"] Jul 06 14:04:27.913209 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:04:27.913182 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33fe9478_5f00_4490_ba78_e12e226c18ff.slice/crio-e7bf707c1607a2d58400b98ab39c4d05eadee890610354f0a40d81fa08c89dc3 WatchSource:0}: Error finding container e7bf707c1607a2d58400b98ab39c4d05eadee890610354f0a40d81fa08c89dc3: Status 404 returned error can't find the container with id e7bf707c1607a2d58400b98ab39c4d05eadee890610354f0a40d81fa08c89dc3 Jul 06 14:04:28.544136 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:28.544100 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696bb467c6-59g9z" event={"ID":"33fe9478-5f00-4490-ba78-e12e226c18ff","Type":"ContainerStarted","Data":"a619d85f47c9f9725e0cdf234a8163d751566cd502dc3d560b39690aaf4fc24e"} Jul 06 14:04:28.544521 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:28.544152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-696bb467c6-59g9z" event={"ID":"33fe9478-5f00-4490-ba78-e12e226c18ff","Type":"ContainerStarted","Data":"e7bf707c1607a2d58400b98ab39c4d05eadee890610354f0a40d81fa08c89dc3"} Jul 06 14:04:28.566240 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:28.566173 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-696bb467c6-59g9z" podStartSLOduration=1.56615383 podStartE2EDuration="1.56615383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:04:28.565718117 +0000 UTC m=+219.754767165" watchObservedRunningTime="2026-07-06 14:04:28.56615383 +0000 UTC m=+219.755202878" Jul 06 14:04:37.771839 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:37.771777 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:37.772282 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:37.771869 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:37.776688 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:37.776667 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:38.591378 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:38.591348 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-696bb467c6-59g9z" Jul 06 14:04:38.655725 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:04:38.655693 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-685649778d-46dvx"] Jul 06 14:05:03.676083 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:03.676038 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-685649778d-46dvx" podUID="ac143289-c8b8-4721-9079-665ca189b950" containerName="console" containerID="cri-o://4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68" gracePeriod=15 Jul 06 14:05:03.925066 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:03.925041 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:05:04.068067 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068026 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-service-ca\") pod \"ac143289-c8b8-4721-9079-665ca189b950\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " Jul 06 14:05:04.068278 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068108 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-console-config\") pod \"ac143289-c8b8-4721-9079-665ca189b950\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " Jul 06 14:05:04.068278 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068133 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-oauth-serving-cert\") pod \"ac143289-c8b8-4721-9079-665ca189b950\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " Jul 06 14:05:04.068278 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068180 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-serving-cert\") pod \"ac143289-c8b8-4721-9079-665ca189b950\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " Jul 06 14:05:04.068439 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068365 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-oauth-config\") pod \"ac143289-c8b8-4721-9079-665ca189b950\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " Jul 06 14:05:04.068439 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068414 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac143289-c8b8-4721-9079-665ca189b950" (UID: "ac143289-c8b8-4721-9079-665ca189b950"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:05:04.068439 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068436 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzw5h\" (UniqueName: \"kubernetes.io/projected/ac143289-c8b8-4721-9079-665ca189b950-kube-api-access-vzw5h\") pod \"ac143289-c8b8-4721-9079-665ca189b950\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " Jul 06 14:05:04.068589 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068482 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-trusted-ca-bundle\") pod \"ac143289-c8b8-4721-9079-665ca189b950\" (UID: \"ac143289-c8b8-4721-9079-665ca189b950\") " Jul 06 14:05:04.068589 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068571 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac143289-c8b8-4721-9079-665ca189b950" (UID: "ac143289-c8b8-4721-9079-665ca189b950"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:05:04.068589 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068577 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-console-config" (OuterVolumeSpecName: "console-config") pod "ac143289-c8b8-4721-9079-665ca189b950" (UID: "ac143289-c8b8-4721-9079-665ca189b950"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:05:04.068955 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068930 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-console-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:05:04.068955 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068949 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ac143289-c8b8-4721-9079-665ca189b950" (UID: "ac143289-c8b8-4721-9079-665ca189b950"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:05:04.069165 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068956 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-oauth-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:05:04.069165 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.068995 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-service-ca\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:05:04.070379 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.070359 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac143289-c8b8-4721-9079-665ca189b950" (UID: "ac143289-c8b8-4721-9079-665ca189b950"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:05:04.070749 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.070723 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac143289-c8b8-4721-9079-665ca189b950" (UID: "ac143289-c8b8-4721-9079-665ca189b950"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:05:04.070749 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.070734 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac143289-c8b8-4721-9079-665ca189b950-kube-api-access-vzw5h" (OuterVolumeSpecName: "kube-api-access-vzw5h") pod "ac143289-c8b8-4721-9079-665ca189b950" (UID: "ac143289-c8b8-4721-9079-665ca189b950"). InnerVolumeSpecName "kube-api-access-vzw5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:05:04.170196 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.170160 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-serving-cert\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:05:04.170196 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.170188 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac143289-c8b8-4721-9079-665ca189b950-console-oauth-config\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:05:04.170196 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.170198 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzw5h\" (UniqueName: \"kubernetes.io/projected/ac143289-c8b8-4721-9079-665ca189b950-kube-api-access-vzw5h\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:05:04.170196 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.170207 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac143289-c8b8-4721-9079-665ca189b950-trusted-ca-bundle\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:05:04.694658 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.694622 2562 generic.go:356] "Generic (PLEG): container finished" podID="ac143289-c8b8-4721-9079-665ca189b950" containerID="4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68" exitCode=0 Jul 06 14:05:04.695079 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.694671 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-685649778d-46dvx" Jul 06 14:05:04.695079 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.694703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685649778d-46dvx" event={"ID":"ac143289-c8b8-4721-9079-665ca189b950","Type":"ContainerDied","Data":"4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68"} Jul 06 14:05:04.695079 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.694758 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-685649778d-46dvx" event={"ID":"ac143289-c8b8-4721-9079-665ca189b950","Type":"ContainerDied","Data":"778595c918f933cccea8f09fe1fd23b7ec8cb4bff610ddbd83361f8a80c74e5b"} Jul 06 14:05:04.695079 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.694783 2562 scope.go:117] "RemoveContainer" containerID="4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68" Jul 06 14:05:04.703305 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.703291 2562 scope.go:117] "RemoveContainer" containerID="4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68" Jul 06 14:05:04.703546 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:05:04.703528 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68\": container with ID starting with 4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68 not found: ID does not exist" containerID="4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68" Jul 06 14:05:04.703601 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.703555 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68"} err="failed to get container status \"4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68\": rpc error: code = NotFound desc = could not find container \"4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68\": container with ID starting with 4d2ba1bec8b9afa4f666ee4006d6b792c97192851fcb0c7561b12c56f608be68 not found: ID does not exist" Jul 06 14:05:04.717949 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.717923 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-685649778d-46dvx"] Jul 06 14:05:04.722195 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:04.722174 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-685649778d-46dvx"] Jul 06 14:05:05.390769 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:05.390726 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac143289-c8b8-4721-9079-665ca189b950" path="/var/lib/kubelet/pods/ac143289-c8b8-4721-9079-665ca189b950/volumes" Jul 06 14:05:49.237573 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:49.237536 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:05:49.241397 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:49.241375 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:05:49.244239 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:05:49.244216 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 14:06:29.335868 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.335829 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2"] Jul 06 14:06:29.336373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.336178 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac143289-c8b8-4721-9079-665ca189b950" containerName="console" Jul 06 14:06:29.336373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.336189 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac143289-c8b8-4721-9079-665ca189b950" containerName="console" Jul 06 14:06:29.336373 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.336254 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac143289-c8b8-4721-9079-665ca189b950" containerName="console" Jul 06 14:06:29.339319 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.339302 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.343388 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.343365 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 14:06:29.344703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.344683 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jbmtl\"" Jul 06 14:06:29.344843 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.344713 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:06:29.349966 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.349942 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2"] Jul 06 14:06:29.433610 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.433579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0364c89b-5576-4441-8e1f-79c3af2f0e5e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2djb2\" (UID: \"0364c89b-5576-4441-8e1f-79c3af2f0e5e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.433804 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.433649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwcss\" (UniqueName: \"kubernetes.io/projected/0364c89b-5576-4441-8e1f-79c3af2f0e5e-kube-api-access-cwcss\") pod \"openshift-lws-operator-fd8ccff4c-2djb2\" (UID: \"0364c89b-5576-4441-8e1f-79c3af2f0e5e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.534693 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.534658 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0364c89b-5576-4441-8e1f-79c3af2f0e5e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2djb2\" (UID: \"0364c89b-5576-4441-8e1f-79c3af2f0e5e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.534857 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.534725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwcss\" (UniqueName: \"kubernetes.io/projected/0364c89b-5576-4441-8e1f-79c3af2f0e5e-kube-api-access-cwcss\") pod \"openshift-lws-operator-fd8ccff4c-2djb2\" (UID: \"0364c89b-5576-4441-8e1f-79c3af2f0e5e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.535119 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.535096 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0364c89b-5576-4441-8e1f-79c3af2f0e5e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2djb2\" (UID: \"0364c89b-5576-4441-8e1f-79c3af2f0e5e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.545133 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.545107 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwcss\" (UniqueName: \"kubernetes.io/projected/0364c89b-5576-4441-8e1f-79c3af2f0e5e-kube-api-access-cwcss\") pod \"openshift-lws-operator-fd8ccff4c-2djb2\" (UID: \"0364c89b-5576-4441-8e1f-79c3af2f0e5e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.655954 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.655852 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" Jul 06 14:06:29.787975 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.787946 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2"] Jul 06 14:06:29.790543 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:06:29.790513 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0364c89b_5576_4441_8e1f_79c3af2f0e5e.slice/crio-8c4217124a5d1f9353b78f9fbf03a80a27b4f53a475563c057575969b035fcbf WatchSource:0}: Error finding container 8c4217124a5d1f9353b78f9fbf03a80a27b4f53a475563c057575969b035fcbf: Status 404 returned error can't find the container with id 8c4217124a5d1f9353b78f9fbf03a80a27b4f53a475563c057575969b035fcbf Jul 06 14:06:29.792072 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:29.792053 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:06:30.036040 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:30.036008 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" event={"ID":"0364c89b-5576-4441-8e1f-79c3af2f0e5e","Type":"ContainerStarted","Data":"8c4217124a5d1f9353b78f9fbf03a80a27b4f53a475563c057575969b035fcbf"} Jul 06 14:06:34.055013 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:34.054971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" event={"ID":"0364c89b-5576-4441-8e1f-79c3af2f0e5e","Type":"ContainerStarted","Data":"8b811cfd1fb10137acf9ff376fc37ece4e801290093c0f1c5c127a57921e5bca"} Jul 06 14:06:34.075586 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:34.075524 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2djb2" podStartSLOduration=1.519080619 podStartE2EDuration="5.075503562s" totalImagesPullingTime="3.556422943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:06:34.07371645 +0000 UTC m=+345.262765492" watchObservedRunningTime="2026-07-06 14:06:34.075503562 +0000 UTC m=+345.264552610" Jul 06 14:06:49.544360 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.544280 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq"] Jul 06 14:06:49.548253 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.548232 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.551247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.551226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2j8qd\"" Jul 06 14:06:49.551350 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.551256 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 14:06:49.551504 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.551488 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 14:06:49.551576 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.551536 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 14:06:49.551576 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.551549 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 14:06:49.569240 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.569201 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq"] Jul 06 14:06:49.719927 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.719878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.719927 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.719931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-manifests\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.720149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.720001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-tmp\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.720149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.720061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.720149 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.720089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld4k8\" (UniqueName: \"kubernetes.io/projected/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-kube-api-access-ld4k8\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.821156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.821069 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.821156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.821108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-manifests\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.821156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.821140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-tmp\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.821420 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.821189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.821420 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.821216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld4k8\" (UniqueName: \"kubernetes.io/projected/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-kube-api-access-ld4k8\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.821587 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.821558 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-manifests\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.823515 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.823487 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-tmp\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.823631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.823536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.823693 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.823636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.839681 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.839653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld4k8\" (UniqueName: \"kubernetes.io/projected/bc80f37c-c88a-4e80-9f07-a321b1e8bfb1-kube-api-access-ld4k8\") pod \"opendatahub-operator-controller-manager-6d88dcf597-kjrrq\" (UID: \"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:49.858507 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:49.858484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:50.002646 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:50.002607 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq"] Jul 06 14:06:50.005863 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:06:50.005837 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc80f37c_c88a_4e80_9f07_a321b1e8bfb1.slice/crio-f10ad2fde3796cafe6f6952f44f1522469c58c835acadae1551b0e979a473a8b WatchSource:0}: Error finding container f10ad2fde3796cafe6f6952f44f1522469c58c835acadae1551b0e979a473a8b: Status 404 returned error can't find the container with id f10ad2fde3796cafe6f6952f44f1522469c58c835acadae1551b0e979a473a8b Jul 06 14:06:50.123002 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:50.122892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" event={"ID":"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1","Type":"ContainerStarted","Data":"f10ad2fde3796cafe6f6952f44f1522469c58c835acadae1551b0e979a473a8b"} Jul 06 14:06:54.146190 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:54.146155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" event={"ID":"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1","Type":"ContainerStarted","Data":"2c5f0dbc0d11db1f9eee23013a9df423d596a5fa04129693f002ab8378ba8913"} Jul 06 14:06:55.152913 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:55.152862 2562 generic.go:356] "Generic (PLEG): container finished" podID="bc80f37c-c88a-4e80-9f07-a321b1e8bfb1" containerID="2c5f0dbc0d11db1f9eee23013a9df423d596a5fa04129693f002ab8378ba8913" exitCode=0 Jul 06 14:06:55.153332 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:55.152977 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" event={"ID":"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1","Type":"ContainerDied","Data":"2c5f0dbc0d11db1f9eee23013a9df423d596a5fa04129693f002ab8378ba8913"} Jul 06 14:06:56.159751 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:56.159719 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" event={"ID":"bc80f37c-c88a-4e80-9f07-a321b1e8bfb1","Type":"ContainerStarted","Data":"06c8d9751838c99f03019c23fbcc8ddfbb0079f1b2b6ef8896cd696269bf0364"} Jul 06 14:06:56.160247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:56.159832 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:06:56.195912 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:06:56.195859 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" podStartSLOduration=1.799593768 podStartE2EDuration="7.195844437s" totalImagesPullingTime="4.353769972s" totalInitContainerRuntime="1.042480697s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:06:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:06:56.194786373 +0000 UTC m=+367.383835470" watchObservedRunningTime="2026-07-06 14:06:56.195844437 +0000 UTC m=+367.384893483" Jul 06 14:07:07.167335 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:07.167307 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d88dcf597-kjrrq" Jul 06 14:07:18.877189 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.877152 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7"] Jul 06 14:07:18.880429 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.880406 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.883421 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.883397 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 14:07:18.883421 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.883398 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 14:07:18.883599 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.883431 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 14:07:18.883780 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.883763 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-m4jqf\"" Jul 06 14:07:18.892939 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.892912 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7"] Jul 06 14:07:18.998526 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998631 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f86e98-10ae-4a13-9639-c367f911d3d8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9dzj\" (UniqueName: \"kubernetes.io/projected/42f86e98-10ae-4a13-9639-c367f911d3d8-kube-api-access-q9dzj\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:18.998933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:18.998780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.057775 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.057740 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f"] Jul 06 14:07:19.060330 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.060311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.063008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.062975 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h444w\"" Jul 06 14:07:19.063008 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.062989 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 14:07:19.063200 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.063050 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 14:07:19.063267 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.063253 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 14:07:19.083092 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.083068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f"] Jul 06 14:07:19.099478 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099655 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099655 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f86e98-10ae-4a13-9639-c367f911d3d8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099655 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099655 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9dzj\" (UniqueName: \"kubernetes.io/projected/42f86e98-10ae-4a13-9639-c367f911d3d8-kube-api-access-q9dzj\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099879 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099879 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099879 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.099879 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099832 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.100090 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.099973 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.100090 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.100063 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.100481 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.100232 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.100481 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.100300 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.100603 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.100568 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42f86e98-10ae-4a13-9639-c367f911d3d8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.102099 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.102077 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.102464 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.102445 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.116492 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.116462 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42f86e98-10ae-4a13-9639-c367f911d3d8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.117681 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.117662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9dzj\" (UniqueName: \"kubernetes.io/projected/42f86e98-10ae-4a13-9639-c367f911d3d8-kube-api-access-q9dzj\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gbst7\" (UID: \"42f86e98-10ae-4a13-9639-c367f911d3d8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.191556 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.191473 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:19.200891 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.200854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a134ed3b-b57c-4d88-a826-2a9172c673bf-manager-config\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.201051 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.200930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a134ed3b-b57c-4d88-a826-2a9172c673bf-cert\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.201051 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.200979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5pk\" (UniqueName: \"kubernetes.io/projected/a134ed3b-b57c-4d88-a826-2a9172c673bf-kube-api-access-rk5pk\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.201162 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.201051 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a134ed3b-b57c-4d88-a826-2a9172c673bf-metrics-cert\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.302593 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.302557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a134ed3b-b57c-4d88-a826-2a9172c673bf-metrics-cert\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.302773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.302686 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a134ed3b-b57c-4d88-a826-2a9172c673bf-manager-config\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.302773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.302727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a134ed3b-b57c-4d88-a826-2a9172c673bf-cert\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.302857 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.302777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5pk\" (UniqueName: \"kubernetes.io/projected/a134ed3b-b57c-4d88-a826-2a9172c673bf-kube-api-access-rk5pk\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.303436 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.303407 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a134ed3b-b57c-4d88-a826-2a9172c673bf-manager-config\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.305188 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.305157 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a134ed3b-b57c-4d88-a826-2a9172c673bf-metrics-cert\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.305790 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.305773 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a134ed3b-b57c-4d88-a826-2a9172c673bf-cert\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.314331 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.314304 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5pk\" (UniqueName: \"kubernetes.io/projected/a134ed3b-b57c-4d88-a826-2a9172c673bf-kube-api-access-rk5pk\") pod \"lws-controller-manager-7456975fc5-8qt4f\" (UID: \"a134ed3b-b57c-4d88-a826-2a9172c673bf\") " pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.332426 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.332402 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7"] Jul 06 14:07:19.336258 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:07:19.336226 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f86e98_10ae_4a13_9639_c367f911d3d8.slice/crio-11f789f725797fa580be1ad69fc801de61e2f160f557da38f3de4e030d6b0f03 WatchSource:0}: Error finding container 11f789f725797fa580be1ad69fc801de61e2f160f557da38f3de4e030d6b0f03: Status 404 returned error can't find the container with id 11f789f725797fa580be1ad69fc801de61e2f160f557da38f3de4e030d6b0f03 Jul 06 14:07:19.371360 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.371331 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:19.542639 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:19.542566 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f"] Jul 06 14:07:19.549221 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:07:19.549156 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda134ed3b_b57c_4d88_a826_2a9172c673bf.slice/crio-14a1c7ef8052ad724921e456cfce783a9bea8b2b54211f1e15c80529e2f4b004 WatchSource:0}: Error finding container 14a1c7ef8052ad724921e456cfce783a9bea8b2b54211f1e15c80529e2f4b004: Status 404 returned error can't find the container with id 14a1c7ef8052ad724921e456cfce783a9bea8b2b54211f1e15c80529e2f4b004 Jul 06 14:07:20.274988 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:20.274878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" event={"ID":"42f86e98-10ae-4a13-9639-c367f911d3d8","Type":"ContainerStarted","Data":"11f789f725797fa580be1ad69fc801de61e2f160f557da38f3de4e030d6b0f03"} Jul 06 14:07:20.276662 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:20.276628 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" event={"ID":"a134ed3b-b57c-4d88-a826-2a9172c673bf","Type":"ContainerStarted","Data":"14a1c7ef8052ad724921e456cfce783a9bea8b2b54211f1e15c80529e2f4b004"} Jul 06 14:07:21.039278 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.039245 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m"] Jul 06 14:07:21.042742 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.042714 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.046099 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.045932 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4787n\"" Jul 06 14:07:21.046607 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.046433 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 14:07:21.055933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.055845 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m"] Jul 06 14:07:21.059763 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.059733 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9"] Jul 06 14:07:21.062859 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.062827 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.066536 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.066379 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 14:07:21.066937 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.066881 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jxv74\"" Jul 06 14:07:21.087773 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.087642 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9"] Jul 06 14:07:21.122335 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.122295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlz4w\" (UniqueName: \"kubernetes.io/projected/e52cb6c2-5732-4193-9f0b-7277f55177fd-kube-api-access-xlz4w\") pod \"kserve-controller-manager-d6f4b65b8-sbz6m\" (UID: \"e52cb6c2-5732-4193-9f0b-7277f55177fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.122539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.122396 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e52cb6c2-5732-4193-9f0b-7277f55177fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-sbz6m\" (UID: \"e52cb6c2-5732-4193-9f0b-7277f55177fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.224118 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.224044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlz4w\" (UniqueName: \"kubernetes.io/projected/e52cb6c2-5732-4193-9f0b-7277f55177fd-kube-api-access-xlz4w\") pod \"kserve-controller-manager-d6f4b65b8-sbz6m\" (UID: \"e52cb6c2-5732-4193-9f0b-7277f55177fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.224118 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.224095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5693d981-477d-458d-a525-938e6f8724f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d8vl9\" (UID: \"5693d981-477d-458d-a525-938e6f8724f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.224303 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.224162 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e52cb6c2-5732-4193-9f0b-7277f55177fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-sbz6m\" (UID: \"e52cb6c2-5732-4193-9f0b-7277f55177fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.224303 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.224189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwjhw\" (UniqueName: \"kubernetes.io/projected/5693d981-477d-458d-a525-938e6f8724f9-kube-api-access-kwjhw\") pod \"llmisvc-controller-manager-777c6bd7d5-d8vl9\" (UID: \"5693d981-477d-458d-a525-938e6f8724f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.224509 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:07:21.224485 2562 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 06 14:07:21.224590 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:07:21.224569 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e52cb6c2-5732-4193-9f0b-7277f55177fd-cert podName:e52cb6c2-5732-4193-9f0b-7277f55177fd nodeName:}" failed. No retries permitted until 2026-07-06 14:07:21.724546302 +0000 UTC m=+392.913595335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e52cb6c2-5732-4193-9f0b-7277f55177fd-cert") pod "kserve-controller-manager-d6f4b65b8-sbz6m" (UID: "e52cb6c2-5732-4193-9f0b-7277f55177fd") : secret "kserve-webhook-server-cert" not found Jul 06 14:07:21.253220 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.253184 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlz4w\" (UniqueName: \"kubernetes.io/projected/e52cb6c2-5732-4193-9f0b-7277f55177fd-kube-api-access-xlz4w\") pod \"kserve-controller-manager-d6f4b65b8-sbz6m\" (UID: \"e52cb6c2-5732-4193-9f0b-7277f55177fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.284669 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.284608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" event={"ID":"a134ed3b-b57c-4d88-a826-2a9172c673bf","Type":"ContainerStarted","Data":"1203f57cac2c09bc7c498fb688734bdc4ce46a88bbdaa46173bd0814c404ea97"} Jul 06 14:07:21.285150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.284937 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:21.320848 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.320729 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" podStartSLOduration=0.675980496 podStartE2EDuration="2.3207078s" totalImagesPullingTime="1.644727304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:07:21.31625947 +0000 UTC m=+392.505308546" watchObservedRunningTime="2026-07-06 14:07:21.3207078 +0000 UTC m=+392.509756849" Jul 06 14:07:21.325023 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.324990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5693d981-477d-458d-a525-938e6f8724f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d8vl9\" (UID: \"5693d981-477d-458d-a525-938e6f8724f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.325208 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.325060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwjhw\" (UniqueName: \"kubernetes.io/projected/5693d981-477d-458d-a525-938e6f8724f9-kube-api-access-kwjhw\") pod \"llmisvc-controller-manager-777c6bd7d5-d8vl9\" (UID: \"5693d981-477d-458d-a525-938e6f8724f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.327701 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.327671 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5693d981-477d-458d-a525-938e6f8724f9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d8vl9\" (UID: \"5693d981-477d-458d-a525-938e6f8724f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.337852 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.337823 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwjhw\" (UniqueName: \"kubernetes.io/projected/5693d981-477d-458d-a525-938e6f8724f9-kube-api-access-kwjhw\") pod \"llmisvc-controller-manager-777c6bd7d5-d8vl9\" (UID: \"5693d981-477d-458d-a525-938e6f8724f9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.377201 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.377147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:21.729579 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.729490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e52cb6c2-5732-4193-9f0b-7277f55177fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-sbz6m\" (UID: \"e52cb6c2-5732-4193-9f0b-7277f55177fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.732075 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.732051 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e52cb6c2-5732-4193-9f0b-7277f55177fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-sbz6m\" (UID: \"e52cb6c2-5732-4193-9f0b-7277f55177fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:21.958324 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:21.958281 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:22.362657 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:22.362614 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:07:22.363085 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:22.362706 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:07:22.363085 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:22.362751 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:07:22.499958 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:22.499927 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9"] Jul 06 14:07:22.504706 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:07:22.504655 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5693d981_477d_458d_a525_938e6f8724f9.slice/crio-e7d927fb43ef2ef7d11299a02fd52d5f936411baee9e4337c6d7854579e571a7 WatchSource:0}: Error finding container e7d927fb43ef2ef7d11299a02fd52d5f936411baee9e4337c6d7854579e571a7: Status 404 returned error can't find the container with id e7d927fb43ef2ef7d11299a02fd52d5f936411baee9e4337c6d7854579e571a7 Jul 06 14:07:22.525775 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:22.522692 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m"] Jul 06 14:07:23.299489 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:23.299445 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" event={"ID":"5693d981-477d-458d-a525-938e6f8724f9","Type":"ContainerStarted","Data":"e7d927fb43ef2ef7d11299a02fd52d5f936411baee9e4337c6d7854579e571a7"} Jul 06 14:07:23.302275 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:23.302207 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" event={"ID":"42f86e98-10ae-4a13-9639-c367f911d3d8","Type":"ContainerStarted","Data":"0652665f8a1ccf171185479707a66ca5f4fbc9e3693bf59d72e3feac3c4dbeda"} Jul 06 14:07:23.303703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:23.303672 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" event={"ID":"e52cb6c2-5732-4193-9f0b-7277f55177fd","Type":"ContainerStarted","Data":"86f6217a22eafa4e2a672e9b5a07a06377a4e3925857ff476946ac4788fbc0a6"} Jul 06 14:07:24.192402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:24.192360 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:24.197697 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:24.197668 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:24.222857 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:24.222803 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" podStartSLOduration=3.199799065 podStartE2EDuration="6.222786643s" totalImagesPullingTime="3.022987578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:07:23.330539264 +0000 UTC m=+394.519588361" watchObservedRunningTime="2026-07-06 14:07:24.222786643 +0000 UTC m=+395.411835692" Jul 06 14:07:24.311234 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:24.311189 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:24.312355 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:24.312332 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gbst7" Jul 06 14:07:26.323470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:26.323373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" event={"ID":"5693d981-477d-458d-a525-938e6f8724f9","Type":"ContainerStarted","Data":"dc92b0b92ee07051a35e40bd06e3088b8948df99543ebbfa23a73c711e286c62"} Jul 06 14:07:26.323470 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:26.323435 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:26.324889 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:26.324867 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" event={"ID":"e52cb6c2-5732-4193-9f0b-7277f55177fd","Type":"ContainerStarted","Data":"e49f6ac03d8eea894dcaeb6f5c4cbff1006c52b0fe5dfc23ae6371c1432fa353"} Jul 06 14:07:26.324983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:26.324945 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:07:26.345703 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:26.345651 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" podStartSLOduration=1.8120687260000001 podStartE2EDuration="5.345635502s" totalImagesPullingTime="3.533566776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:07:26.343335821 +0000 UTC m=+397.532384891" watchObservedRunningTime="2026-07-06 14:07:26.345635502 +0000 UTC m=+397.534684549" Jul 06 14:07:26.362344 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:26.362279 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" podStartSLOduration=1.857992269 podStartE2EDuration="5.3622574s" totalImagesPullingTime="3.504265131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:07:26.361252827 +0000 UTC m=+397.550301899" watchObservedRunningTime="2026-07-06 14:07:26.3622574 +0000 UTC m=+397.551306448" Jul 06 14:07:32.292319 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:32.292283 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7456975fc5-8qt4f" Jul 06 14:07:57.336770 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:57.332073 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d8vl9" Jul 06 14:07:57.336770 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:07:57.336499 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-sbz6m" Jul 06 14:08:11.511950 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.511855 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-skrq4"] Jul 06 14:08:11.514204 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.514188 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" Jul 06 14:08:11.517034 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.517015 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 14:08:11.518280 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.518260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 14:08:11.518399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.518265 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-x4zvz\"" Jul 06 14:08:11.527981 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.527949 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-skrq4"] Jul 06 14:08:11.605271 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.605236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2jv4\" (UniqueName: \"kubernetes.io/projected/e183c07c-5a60-498f-8617-6a9f3ab0b3a7-kube-api-access-h2jv4\") pod \"kuadrant-operator-catalog-skrq4\" (UID: \"e183c07c-5a60-498f-8617-6a9f3ab0b3a7\") " pod="kuadrant-system/kuadrant-operator-catalog-skrq4" Jul 06 14:08:11.705930 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.705879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2jv4\" (UniqueName: \"kubernetes.io/projected/e183c07c-5a60-498f-8617-6a9f3ab0b3a7-kube-api-access-h2jv4\") pod \"kuadrant-operator-catalog-skrq4\" (UID: \"e183c07c-5a60-498f-8617-6a9f3ab0b3a7\") " pod="kuadrant-system/kuadrant-operator-catalog-skrq4" Jul 06 14:08:11.715516 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.715486 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2jv4\" (UniqueName: \"kubernetes.io/projected/e183c07c-5a60-498f-8617-6a9f3ab0b3a7-kube-api-access-h2jv4\") pod \"kuadrant-operator-catalog-skrq4\" (UID: \"e183c07c-5a60-498f-8617-6a9f3ab0b3a7\") " pod="kuadrant-system/kuadrant-operator-catalog-skrq4" Jul 06 14:08:11.824588 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.824493 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" Jul 06 14:08:11.846726 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.846698 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-skrq4"] Jul 06 14:08:11.960345 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:11.960308 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-skrq4"] Jul 06 14:08:11.962306 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:08:11.962282 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode183c07c_5a60_498f_8617_6a9f3ab0b3a7.slice/crio-a888e92a2133ba68f5e88e13498fbe47982453ab3c9e053886902086c4907bd3 WatchSource:0}: Error finding container a888e92a2133ba68f5e88e13498fbe47982453ab3c9e053886902086c4907bd3: Status 404 returned error can't find the container with id a888e92a2133ba68f5e88e13498fbe47982453ab3c9e053886902086c4907bd3 Jul 06 14:08:12.056180 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.056149 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdscf"] Jul 06 14:08:12.059257 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.059238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:12.070540 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.070512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdscf"] Jul 06 14:08:12.109522 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.109426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45lph\" (UniqueName: \"kubernetes.io/projected/ef4b7ce5-b65d-4381-a6ab-89dc62e908f8-kube-api-access-45lph\") pod \"kuadrant-operator-catalog-jdscf\" (UID: \"ef4b7ce5-b65d-4381-a6ab-89dc62e908f8\") " pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:12.210223 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.210181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45lph\" (UniqueName: \"kubernetes.io/projected/ef4b7ce5-b65d-4381-a6ab-89dc62e908f8-kube-api-access-45lph\") pod \"kuadrant-operator-catalog-jdscf\" (UID: \"ef4b7ce5-b65d-4381-a6ab-89dc62e908f8\") " pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:12.221740 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.221712 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45lph\" (UniqueName: \"kubernetes.io/projected/ef4b7ce5-b65d-4381-a6ab-89dc62e908f8-kube-api-access-45lph\") pod \"kuadrant-operator-catalog-jdscf\" (UID: \"ef4b7ce5-b65d-4381-a6ab-89dc62e908f8\") " pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:12.368933 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.368828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:12.499454 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.499429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jdscf"] Jul 06 14:08:12.501608 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:08:12.501581 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef4b7ce5_b65d_4381_a6ab_89dc62e908f8.slice/crio-d3c07fb756ce2b2a78da1d913d05ace3935a9b0c3ef2498d2358ef6c644558fd WatchSource:0}: Error finding container d3c07fb756ce2b2a78da1d913d05ace3935a9b0c3ef2498d2358ef6c644558fd: Status 404 returned error can't find the container with id d3c07fb756ce2b2a78da1d913d05ace3935a9b0c3ef2498d2358ef6c644558fd Jul 06 14:08:12.553270 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.553231 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" event={"ID":"ef4b7ce5-b65d-4381-a6ab-89dc62e908f8","Type":"ContainerStarted","Data":"d3c07fb756ce2b2a78da1d913d05ace3935a9b0c3ef2498d2358ef6c644558fd"} Jul 06 14:08:12.554422 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:12.554395 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" event={"ID":"e183c07c-5a60-498f-8617-6a9f3ab0b3a7","Type":"ContainerStarted","Data":"a888e92a2133ba68f5e88e13498fbe47982453ab3c9e053886902086c4907bd3"} Jul 06 14:08:14.571356 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.571312 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" event={"ID":"ef4b7ce5-b65d-4381-a6ab-89dc62e908f8","Type":"ContainerStarted","Data":"7d21c2fa651b37580a645a2680ec85922c20ec7dd49845a1e2f9ddd59a0632e6"} Jul 06 14:08:14.572848 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.572810 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" event={"ID":"e183c07c-5a60-498f-8617-6a9f3ab0b3a7","Type":"ContainerStarted","Data":"f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789"} Jul 06 14:08:14.572988 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.572864 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" podUID="e183c07c-5a60-498f-8617-6a9f3ab0b3a7" containerName="registry-server" containerID="cri-o://f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789" gracePeriod=2 Jul 06 14:08:14.613843 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.613790 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" podStartSLOduration=0.77120076 podStartE2EDuration="2.613774588s" totalImagesPullingTime="1.842573828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:08:14.597965511 +0000 UTC m=+445.787014576" watchObservedRunningTime="2026-07-06 14:08:14.613774588 +0000 UTC m=+445.802823636" Jul 06 14:08:14.626985 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.626931 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" podStartSLOduration=1.474317814 podStartE2EDuration="3.626917384s" totalImagesPullingTime="2.15259957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:08:14.625948045 +0000 UTC m=+445.814997092" watchObservedRunningTime="2026-07-06 14:08:14.626917384 +0000 UTC m=+445.815966429" Jul 06 14:08:14.814631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.814608 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" Jul 06 14:08:14.838038 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.838012 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2jv4\" (UniqueName: \"kubernetes.io/projected/e183c07c-5a60-498f-8617-6a9f3ab0b3a7-kube-api-access-h2jv4\") pod \"e183c07c-5a60-498f-8617-6a9f3ab0b3a7\" (UID: \"e183c07c-5a60-498f-8617-6a9f3ab0b3a7\") " Jul 06 14:08:14.840206 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.840181 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e183c07c-5a60-498f-8617-6a9f3ab0b3a7-kube-api-access-h2jv4" (OuterVolumeSpecName: "kube-api-access-h2jv4") pod "e183c07c-5a60-498f-8617-6a9f3ab0b3a7" (UID: "e183c07c-5a60-498f-8617-6a9f3ab0b3a7"). InnerVolumeSpecName "kube-api-access-h2jv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:08:14.939398 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:14.939308 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2jv4\" (UniqueName: \"kubernetes.io/projected/e183c07c-5a60-498f-8617-6a9f3ab0b3a7-kube-api-access-h2jv4\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:08:15.579242 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.579209 2562 generic.go:356] "Generic (PLEG): container finished" podID="e183c07c-5a60-498f-8617-6a9f3ab0b3a7" containerID="f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789" exitCode=0 Jul 06 14:08:15.579688 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.579253 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" Jul 06 14:08:15.579688 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.579300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" event={"ID":"e183c07c-5a60-498f-8617-6a9f3ab0b3a7","Type":"ContainerDied","Data":"f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789"} Jul 06 14:08:15.579688 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.579344 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-skrq4" event={"ID":"e183c07c-5a60-498f-8617-6a9f3ab0b3a7","Type":"ContainerDied","Data":"a888e92a2133ba68f5e88e13498fbe47982453ab3c9e053886902086c4907bd3"} Jul 06 14:08:15.579688 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.579363 2562 scope.go:117] "RemoveContainer" containerID="f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789" Jul 06 14:08:15.588144 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.588124 2562 scope.go:117] "RemoveContainer" containerID="f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789" Jul 06 14:08:15.588427 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:08:15.588406 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789\": container with ID starting with f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789 not found: ID does not exist" containerID="f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789" Jul 06 14:08:15.588474 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.588438 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789"} err="failed to get container status \"f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789\": rpc error: code = NotFound desc = could not find container \"f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789\": container with ID starting with f509279ccae86617a5367b1ab6388eaba4c1c8e84aa338739b6cd8e7aa502789 not found: ID does not exist" Jul 06 14:08:15.604683 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.604658 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-skrq4"] Jul 06 14:08:15.614262 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:15.614234 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-skrq4"] Jul 06 14:08:17.389684 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:17.389646 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e183c07c-5a60-498f-8617-6a9f3ab0b3a7" path="/var/lib/kubelet/pods/e183c07c-5a60-498f-8617-6a9f3ab0b3a7/volumes" Jul 06 14:08:22.368958 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:22.368923 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:22.369444 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:22.368982 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:22.391712 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:22.391683 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:22.637126 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:22.637053 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jdscf" Jul 06 14:08:42.923852 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:42.923818 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r"] Jul 06 14:08:42.924314 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:42.924210 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e183c07c-5a60-498f-8617-6a9f3ab0b3a7" containerName="registry-server" Jul 06 14:08:42.924314 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:42.924221 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e183c07c-5a60-498f-8617-6a9f3ab0b3a7" containerName="registry-server" Jul 06 14:08:42.924314 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:42.924292 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e183c07c-5a60-498f-8617-6a9f3ab0b3a7" containerName="registry-server" Jul 06 14:08:42.927261 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:42.927244 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:08:42.931397 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:42.931375 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ts9lb\"" Jul 06 14:08:42.940374 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:42.940346 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r"] Jul 06 14:08:43.016296 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:43.016254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m54vp\" (UniqueName: \"kubernetes.io/projected/f72d654d-c615-4499-a287-56b1acd88a59-kube-api-access-m54vp\") pod \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" (UID: \"f72d654d-c615-4499-a287-56b1acd88a59\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:08:43.117420 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:43.117377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m54vp\" (UniqueName: \"kubernetes.io/projected/f72d654d-c615-4499-a287-56b1acd88a59-kube-api-access-m54vp\") pod \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" (UID: \"f72d654d-c615-4499-a287-56b1acd88a59\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:08:43.128785 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:43.128739 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m54vp\" (UniqueName: \"kubernetes.io/projected/f72d654d-c615-4499-a287-56b1acd88a59-kube-api-access-m54vp\") pod \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" (UID: \"f72d654d-c615-4499-a287-56b1acd88a59\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:08:43.237399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:43.237297 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:08:43.389496 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:08:43.389465 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72d654d_c615_4499_a287_56b1acd88a59.slice/crio-4d690fd26e642b6bfef2d784374b8775357e933c250dc5aed93cb654a61d66e0 WatchSource:0}: Error finding container 4d690fd26e642b6bfef2d784374b8775357e933c250dc5aed93cb654a61d66e0: Status 404 returned error can't find the container with id 4d690fd26e642b6bfef2d784374b8775357e933c250dc5aed93cb654a61d66e0 Jul 06 14:08:43.390652 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:43.390627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r"] Jul 06 14:08:43.711775 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:43.711741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" event={"ID":"f72d654d-c615-4499-a287-56b1acd88a59","Type":"ContainerStarted","Data":"4d690fd26e642b6bfef2d784374b8775357e933c250dc5aed93cb654a61d66e0"} Jul 06 14:08:45.722779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:45.722687 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" event={"ID":"f72d654d-c615-4499-a287-56b1acd88a59","Type":"ContainerStarted","Data":"2aa9c66344a737680146bea10e4ddfad43b87718d6867730ba0a7f6ee88e0f1a"} Jul 06 14:08:45.722779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:45.722771 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:08:45.748596 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:45.748544 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" podStartSLOduration=1.708644796 podStartE2EDuration="3.748528463s" totalImagesPullingTime="2.039883667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:08:45.745630455 +0000 UTC m=+476.934679519" watchObservedRunningTime="2026-07-06 14:08:45.748528463 +0000 UTC m=+476.937577509" Jul 06 14:08:52.164876 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.164839 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng"] Jul 06 14:08:52.171555 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.171537 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.174510 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.174491 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mp4q4\"" Jul 06 14:08:52.183234 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.183209 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng"] Jul 06 14:08:52.206653 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.206616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssvqm\" (UniqueName: \"kubernetes.io/projected/3f7528ba-5523-4023-8175-41469ee52e06-kube-api-access-ssvqm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.206820 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.206760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f7528ba-5523-4023-8175-41469ee52e06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.307862 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.307820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f7528ba-5523-4023-8175-41469ee52e06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.307862 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.307868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssvqm\" (UniqueName: \"kubernetes.io/projected/3f7528ba-5523-4023-8175-41469ee52e06-kube-api-access-ssvqm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.308242 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.308221 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f7528ba-5523-4023-8175-41469ee52e06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.321104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.321066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssvqm\" (UniqueName: \"kubernetes.io/projected/3f7528ba-5523-4023-8175-41469ee52e06-kube-api-access-ssvqm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.483441 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.483351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:52.629521 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.629495 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng"] Jul 06 14:08:52.631819 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:08:52.631788 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7528ba_5523_4023_8175_41469ee52e06.slice/crio-5a2e1f0518ef40c799f29b6719de8c3d2b3111056b77b8de327f784cab0dcd90 WatchSource:0}: Error finding container 5a2e1f0518ef40c799f29b6719de8c3d2b3111056b77b8de327f784cab0dcd90: Status 404 returned error can't find the container with id 5a2e1f0518ef40c799f29b6719de8c3d2b3111056b77b8de327f784cab0dcd90 Jul 06 14:08:52.759838 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:52.759745 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" event={"ID":"3f7528ba-5523-4023-8175-41469ee52e06","Type":"ContainerStarted","Data":"5a2e1f0518ef40c799f29b6719de8c3d2b3111056b77b8de327f784cab0dcd90"} Jul 06 14:08:56.730370 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:56.730338 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:08:59.799526 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:59.799491 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" event={"ID":"3f7528ba-5523-4023-8175-41469ee52e06","Type":"ContainerStarted","Data":"8f73e3086d0c86802c4f4d253e9791b3ab8fd804427c650834bc79218b82c297"} Jul 06 14:08:59.799973 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:59.799584 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:08:59.822956 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:08:59.822882 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" podStartSLOduration=1.5599950470000001 podStartE2EDuration="7.822862925s" totalImagesPullingTime="6.262867878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:08:59.821961836 +0000 UTC m=+491.011010901" watchObservedRunningTime="2026-07-06 14:08:59.822862925 +0000 UTC m=+491.011912045" Jul 06 14:09:10.806706 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:10.806675 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:12.584733 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.584688 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng"] Jul 06 14:09:12.585207 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.584989 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" podUID="3f7528ba-5523-4023-8175-41469ee52e06" containerName="manager" containerID="cri-o://8f73e3086d0c86802c4f4d253e9791b3ab8fd804427c650834bc79218b82c297" gracePeriod=2 Jul 06 14:09:12.591429 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.591401 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng"] Jul 06 14:09:12.620627 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.620592 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r"] Jul 06 14:09:12.620890 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.620859 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" podUID="f72d654d-c615-4499-a287-56b1acd88a59" containerName="manager" containerID="cri-o://2aa9c66344a737680146bea10e4ddfad43b87718d6867730ba0a7f6ee88e0f1a" gracePeriod=2 Jul 06 14:09:12.624170 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.624114 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r"] Jul 06 14:09:12.626200 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.626162 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="f72d654d-c615-4499-a287-56b1acd88a59" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:09:12.630037 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.630003 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="3f7528ba-5523-4023-8175-41469ee52e06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:12.633949 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.633925 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp"] Jul 06 14:09:12.634345 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.634330 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f7528ba-5523-4023-8175-41469ee52e06" containerName="manager" Jul 06 14:09:12.634423 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.634348 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f7528ba-5523-4023-8175-41469ee52e06" containerName="manager" Jul 06 14:09:12.634423 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.634373 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f72d654d-c615-4499-a287-56b1acd88a59" containerName="manager" Jul 06 14:09:12.634423 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.634382 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f72d654d-c615-4499-a287-56b1acd88a59" containerName="manager" Jul 06 14:09:12.634576 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.634498 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f72d654d-c615-4499-a287-56b1acd88a59" containerName="manager" Jul 06 14:09:12.634576 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.634514 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f7528ba-5523-4023-8175-41469ee52e06" containerName="manager" Jul 06 14:09:12.637771 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.637749 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:12.673788 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.673753 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp"] Jul 06 14:09:12.675599 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.675571 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2"] Jul 06 14:09:12.678760 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.678741 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" Jul 06 14:09:12.696402 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.696363 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="f72d654d-c615-4499-a287-56b1acd88a59" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:09:12.698647 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.698621 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2"] Jul 06 14:09:12.698969 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.698938 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="3f7528ba-5523-4023-8175-41469ee52e06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:12.701610 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.701582 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="f72d654d-c615-4499-a287-56b1acd88a59" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:09:12.734406 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.734372 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="3f7528ba-5523-4023-8175-41469ee52e06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:12.804970 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.804855 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hrkvp\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:12.805151 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.805039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44pbf\" (UniqueName: \"kubernetes.io/projected/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-kube-api-access-44pbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hrkvp\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:12.805151 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.805087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcff7\" (UniqueName: \"kubernetes.io/projected/69dcbb7c-3c67-43dc-827b-1869518d3c26-kube-api-access-qcff7\") pod \"limitador-operator-controller-manager-859d7b78bd-ldld2\" (UID: \"69dcbb7c-3c67-43dc-827b-1869518d3c26\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" Jul 06 14:09:12.867350 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.867325 2562 generic.go:356] "Generic (PLEG): container finished" podID="f72d654d-c615-4499-a287-56b1acd88a59" containerID="2aa9c66344a737680146bea10e4ddfad43b87718d6867730ba0a7f6ee88e0f1a" exitCode=0 Jul 06 14:09:12.867477 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.867411 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d690fd26e642b6bfef2d784374b8775357e933c250dc5aed93cb654a61d66e0" Jul 06 14:09:12.868829 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.868797 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f7528ba-5523-4023-8175-41469ee52e06" containerID="8f73e3086d0c86802c4f4d253e9791b3ab8fd804427c650834bc79218b82c297" exitCode=0 Jul 06 14:09:12.868968 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.868870 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a2e1f0518ef40c799f29b6719de8c3d2b3111056b77b8de327f784cab0dcd90" Jul 06 14:09:12.878885 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.878862 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:12.881409 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.881382 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="f72d654d-c615-4499-a287-56b1acd88a59" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:09:12.882978 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.882963 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:09:12.883676 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.883657 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="3f7528ba-5523-4023-8175-41469ee52e06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:12.885842 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.885821 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-p2m2r\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="f72d654d-c615-4499-a287-56b1acd88a59" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:09:12.887891 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:12.887867 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4lgng\" is forbidden: User \"system:node:ip-10-0-129-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-92.ec2.internal' and this object" podUID="3f7528ba-5523-4023-8175-41469ee52e06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:12.906331 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.906304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hrkvp\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:12.906484 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.906336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44pbf\" (UniqueName: \"kubernetes.io/projected/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-kube-api-access-44pbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hrkvp\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:12.906484 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.906358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcff7\" (UniqueName: \"kubernetes.io/projected/69dcbb7c-3c67-43dc-827b-1869518d3c26-kube-api-access-qcff7\") pod \"limitador-operator-controller-manager-859d7b78bd-ldld2\" (UID: \"69dcbb7c-3c67-43dc-827b-1869518d3c26\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" Jul 06 14:09:12.906715 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.906695 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hrkvp\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:12.918960 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.918939 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcff7\" (UniqueName: \"kubernetes.io/projected/69dcbb7c-3c67-43dc-827b-1869518d3c26-kube-api-access-qcff7\") pod \"limitador-operator-controller-manager-859d7b78bd-ldld2\" (UID: \"69dcbb7c-3c67-43dc-827b-1869518d3c26\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" Jul 06 14:09:12.919053 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:12.918984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44pbf\" (UniqueName: \"kubernetes.io/projected/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-kube-api-access-44pbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hrkvp\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:13.007029 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.006994 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:13.007192 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.007105 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f7528ba-5523-4023-8175-41469ee52e06-extensions-socket-volume\") pod \"3f7528ba-5523-4023-8175-41469ee52e06\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " Jul 06 14:09:13.007250 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.007229 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m54vp\" (UniqueName: \"kubernetes.io/projected/f72d654d-c615-4499-a287-56b1acd88a59-kube-api-access-m54vp\") pod \"f72d654d-c615-4499-a287-56b1acd88a59\" (UID: \"f72d654d-c615-4499-a287-56b1acd88a59\") " Jul 06 14:09:13.007298 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.007248 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssvqm\" (UniqueName: \"kubernetes.io/projected/3f7528ba-5523-4023-8175-41469ee52e06-kube-api-access-ssvqm\") pod \"3f7528ba-5523-4023-8175-41469ee52e06\" (UID: \"3f7528ba-5523-4023-8175-41469ee52e06\") " Jul 06 14:09:13.007604 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.007580 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f7528ba-5523-4023-8175-41469ee52e06-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3f7528ba-5523-4023-8175-41469ee52e06" (UID: "3f7528ba-5523-4023-8175-41469ee52e06"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:09:13.009323 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.009300 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f7528ba-5523-4023-8175-41469ee52e06-kube-api-access-ssvqm" (OuterVolumeSpecName: "kube-api-access-ssvqm") pod "3f7528ba-5523-4023-8175-41469ee52e06" (UID: "3f7528ba-5523-4023-8175-41469ee52e06"). InnerVolumeSpecName "kube-api-access-ssvqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:09:13.009425 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.009367 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f72d654d-c615-4499-a287-56b1acd88a59-kube-api-access-m54vp" (OuterVolumeSpecName: "kube-api-access-m54vp") pod "f72d654d-c615-4499-a287-56b1acd88a59" (UID: "f72d654d-c615-4499-a287-56b1acd88a59"). InnerVolumeSpecName "kube-api-access-m54vp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:09:13.013660 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.013195 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" Jul 06 14:09:13.108311 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.108281 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m54vp\" (UniqueName: \"kubernetes.io/projected/f72d654d-c615-4499-a287-56b1acd88a59-kube-api-access-m54vp\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:09:13.108311 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.108307 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssvqm\" (UniqueName: \"kubernetes.io/projected/3f7528ba-5523-4023-8175-41469ee52e06-kube-api-access-ssvqm\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:09:13.108501 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.108320 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f7528ba-5523-4023-8175-41469ee52e06-extensions-socket-volume\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:09:13.160036 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.160004 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp"] Jul 06 14:09:13.163350 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:09:13.163323 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f49e5e3_95f3_499a_92a8_0fa9f14741e7.slice/crio-f96ef0cc1c9424825f234eeebbf91b078558449a351c917a2bb04a23d1ae0bba WatchSource:0}: Error finding container f96ef0cc1c9424825f234eeebbf91b078558449a351c917a2bb04a23d1ae0bba: Status 404 returned error can't find the container with id f96ef0cc1c9424825f234eeebbf91b078558449a351c917a2bb04a23d1ae0bba Jul 06 14:09:13.174582 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.174561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2"] Jul 06 14:09:13.176958 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:09:13.176936 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69dcbb7c_3c67_43dc_827b_1869518d3c26.slice/crio-b71d955d7bb29ab8dc5cca413e3a35d38ad08a4bef3d7d3538125574c5a7972f WatchSource:0}: Error finding container b71d955d7bb29ab8dc5cca413e3a35d38ad08a4bef3d7d3538125574c5a7972f: Status 404 returned error can't find the container with id b71d955d7bb29ab8dc5cca413e3a35d38ad08a4bef3d7d3538125574c5a7972f Jul 06 14:09:13.391288 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.391208 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f7528ba-5523-4023-8175-41469ee52e06" path="/var/lib/kubelet/pods/3f7528ba-5523-4023-8175-41469ee52e06/volumes" Jul 06 14:09:13.391577 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.391564 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f72d654d-c615-4499-a287-56b1acd88a59" path="/var/lib/kubelet/pods/f72d654d-c615-4499-a287-56b1acd88a59/volumes" Jul 06 14:09:13.876624 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.876591 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" event={"ID":"69dcbb7c-3c67-43dc-827b-1869518d3c26","Type":"ContainerStarted","Data":"acc7fab4822e81aeb10a38ea15689513a54415c3899f3b5f57e0010ef99cbdbe"} Jul 06 14:09:13.877086 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.876641 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" event={"ID":"69dcbb7c-3c67-43dc-827b-1869518d3c26","Type":"ContainerStarted","Data":"b71d955d7bb29ab8dc5cca413e3a35d38ad08a4bef3d7d3538125574c5a7972f"} Jul 06 14:09:13.877086 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.876775 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" Jul 06 14:09:13.878371 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.878346 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" event={"ID":"3f49e5e3-95f3-499a-92a8-0fa9f14741e7","Type":"ContainerStarted","Data":"aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af"} Jul 06 14:09:13.878487 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.878389 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p2m2r" Jul 06 14:09:13.878487 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.878391 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:13.878566 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.878506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" event={"ID":"3f49e5e3-95f3-499a-92a8-0fa9f14741e7","Type":"ContainerStarted","Data":"f96ef0cc1c9424825f234eeebbf91b078558449a351c917a2bb04a23d1ae0bba"} Jul 06 14:09:13.878566 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.878391 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4lgng" Jul 06 14:09:13.903608 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.903561 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" podStartSLOduration=1.903548088 podStartE2EDuration="1.903548088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:09:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:09:13.902769537 +0000 UTC m=+505.091818585" watchObservedRunningTime="2026-07-06 14:09:13.903548088 +0000 UTC m=+505.092597133" Jul 06 14:09:13.933037 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:13.932991 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" podStartSLOduration=1.932976187 podStartE2EDuration="1.932976187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:09:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:09:13.931157851 +0000 UTC m=+505.120206922" watchObservedRunningTime="2026-07-06 14:09:13.932976187 +0000 UTC m=+505.122025233" Jul 06 14:09:24.886536 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:24.886502 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ldld2" Jul 06 14:09:24.887122 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:24.886594 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:28.945383 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:28.945346 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp"] Jul 06 14:09:28.945826 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:28.945566 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" podUID="3f49e5e3-95f3-499a-92a8-0fa9f14741e7" containerName="manager" containerID="cri-o://aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af" gracePeriod=10 Jul 06 14:09:29.193726 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.193700 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:29.250618 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.250517 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-extensions-socket-volume\") pod \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " Jul 06 14:09:29.250800 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.250621 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44pbf\" (UniqueName: \"kubernetes.io/projected/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-kube-api-access-44pbf\") pod \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\" (UID: \"3f49e5e3-95f3-499a-92a8-0fa9f14741e7\") " Jul 06 14:09:29.250993 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.250969 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3f49e5e3-95f3-499a-92a8-0fa9f14741e7" (UID: "3f49e5e3-95f3-499a-92a8-0fa9f14741e7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:09:29.252579 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.252558 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-kube-api-access-44pbf" (OuterVolumeSpecName: "kube-api-access-44pbf") pod "3f49e5e3-95f3-499a-92a8-0fa9f14741e7" (UID: "3f49e5e3-95f3-499a-92a8-0fa9f14741e7"). InnerVolumeSpecName "kube-api-access-44pbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:09:29.351872 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.351826 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44pbf\" (UniqueName: \"kubernetes.io/projected/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-kube-api-access-44pbf\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:09:29.351872 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.351861 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f49e5e3-95f3-499a-92a8-0fa9f14741e7-extensions-socket-volume\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:09:29.962810 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.962773 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f49e5e3-95f3-499a-92a8-0fa9f14741e7" containerID="aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af" exitCode=0 Jul 06 14:09:29.963388 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.962816 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" Jul 06 14:09:29.963388 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.962856 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" event={"ID":"3f49e5e3-95f3-499a-92a8-0fa9f14741e7","Type":"ContainerDied","Data":"aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af"} Jul 06 14:09:29.963388 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.962944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp" event={"ID":"3f49e5e3-95f3-499a-92a8-0fa9f14741e7","Type":"ContainerDied","Data":"f96ef0cc1c9424825f234eeebbf91b078558449a351c917a2bb04a23d1ae0bba"} Jul 06 14:09:29.963388 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.962996 2562 scope.go:117] "RemoveContainer" containerID="aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af" Jul 06 14:09:29.971965 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.971930 2562 scope.go:117] "RemoveContainer" containerID="aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af" Jul 06 14:09:29.972221 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:09:29.972202 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af\": container with ID starting with aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af not found: ID does not exist" containerID="aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af" Jul 06 14:09:29.972305 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.972228 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af"} err="failed to get container status \"aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af\": rpc error: code = NotFound desc = could not find container \"aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af\": container with ID starting with aa54684812253b8307e845121c4030b208e5662b37df75bfdaab0dc0b58e48af not found: ID does not exist" Jul 06 14:09:29.996630 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:29.996592 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp"] Jul 06 14:09:30.006555 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:30.006524 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hrkvp"] Jul 06 14:09:31.388545 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:31.388513 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f49e5e3-95f3-499a-92a8-0fa9f14741e7" path="/var/lib/kubelet/pods/3f49e5e3-95f3-499a-92a8-0fa9f14741e7/volumes" Jul 06 14:09:45.517702 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.517625 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb"] Jul 06 14:09:45.518138 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.518120 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f49e5e3-95f3-499a-92a8-0fa9f14741e7" containerName="manager" Jul 06 14:09:45.518138 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.518139 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f49e5e3-95f3-499a-92a8-0fa9f14741e7" containerName="manager" Jul 06 14:09:45.518241 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.518209 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f49e5e3-95f3-499a-92a8-0fa9f14741e7" containerName="manager" Jul 06 14:09:45.520705 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.520683 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.524248 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.524229 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x7w97\"" Jul 06 14:09:45.535684 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.535655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb"] Jul 06 14:09:45.605218 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc4mg\" (UniqueName: \"kubernetes.io/projected/8f349803-9b28-4683-b81f-9bb0f4b0639a-kube-api-access-hc4mg\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605389 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605389 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605389 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605389 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f349803-9b28-4683-b81f-9bb0f4b0639a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605527 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605390 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605527 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605427 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605527 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.605527 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.605486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.706499 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc4mg\" (UniqueName: \"kubernetes.io/projected/8f349803-9b28-4683-b81f-9bb0f4b0639a-kube-api-access-hc4mg\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.706682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.706682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.706682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.706682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f349803-9b28-4683-b81f-9bb0f4b0639a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.706682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.706682 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.707037 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.707037 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.706725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.707142 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.707102 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.707199 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.707150 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.707501 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.707476 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.707596 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.707583 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f349803-9b28-4683-b81f-9bb0f4b0639a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.707691 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.707664 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.709416 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.709397 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.709733 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.709711 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.721573 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.721542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f349803-9b28-4683-b81f-9bb0f4b0639a-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.721699 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.721556 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc4mg\" (UniqueName: \"kubernetes.io/projected/8f349803-9b28-4683-b81f-9bb0f4b0639a-kube-api-access-hc4mg\") pod \"maas-default-gateway-openshift-default-5896d4dd49-nmqnb\" (UID: \"8f349803-9b28-4683-b81f-9bb0f4b0639a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.832107 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.832016 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:45.974760 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.974722 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb"] Jul 06 14:09:45.978109 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:09:45.978085 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f349803_9b28_4683_b81f_9bb0f4b0639a.slice/crio-ec5d16827253891abcb5e678e99602d5bbc6ce55b736f4ff360f93bc285c8adf WatchSource:0}: Error finding container ec5d16827253891abcb5e678e99602d5bbc6ce55b736f4ff360f93bc285c8adf: Status 404 returned error can't find the container with id ec5d16827253891abcb5e678e99602d5bbc6ce55b736f4ff360f93bc285c8adf Jul 06 14:09:45.980169 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.980139 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:09:45.980229 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.980206 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:09:45.980266 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:45.980234 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 14:09:46.045271 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:46.045246 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" event={"ID":"8f349803-9b28-4683-b81f-9bb0f4b0639a","Type":"ContainerStarted","Data":"bc0875500d0cbf71f750faf27c264bbbc73071e00ef31f68e2a8e02c178c6476"} Jul 06 14:09:46.045400 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:46.045305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" event={"ID":"8f349803-9b28-4683-b81f-9bb0f4b0639a","Type":"ContainerStarted","Data":"ec5d16827253891abcb5e678e99602d5bbc6ce55b736f4ff360f93bc285c8adf"} Jul 06 14:09:46.078681 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:46.078630 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" podStartSLOduration=1.07861156 podStartE2EDuration="1.07861156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:09:46.07671316 +0000 UTC m=+537.265762230" watchObservedRunningTime="2026-07-06 14:09:46.07861156 +0000 UTC m=+537.267660608" Jul 06 14:09:46.833193 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:46.833153 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:47.837443 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:47.837412 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:48.057715 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:48.057686 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:48.062989 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:48.062962 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-nmqnb" Jul 06 14:09:59.884070 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:59.884029 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-chpjx"] Jul 06 14:09:59.888386 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:59.888361 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:09:59.894024 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:59.893999 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 14:09:59.894320 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:59.894302 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h4dd8\"" Jul 06 14:09:59.910543 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:59.910511 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-chpjx"] Jul 06 14:09:59.957885 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:09:59.957853 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-chpjx"] Jul 06 14:10:00.042956 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.042875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jttxw\" (UniqueName: \"kubernetes.io/projected/10623230-b62a-40f8-828a-c3008738eed8-kube-api-access-jttxw\") pod \"limitador-limitador-64c94fd64f-chpjx\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:00.043136 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.043015 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10623230-b62a-40f8-828a-c3008738eed8-config-file\") pod \"limitador-limitador-64c94fd64f-chpjx\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:00.143930 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.143813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jttxw\" (UniqueName: \"kubernetes.io/projected/10623230-b62a-40f8-828a-c3008738eed8-kube-api-access-jttxw\") pod \"limitador-limitador-64c94fd64f-chpjx\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:00.143930 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.143890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10623230-b62a-40f8-828a-c3008738eed8-config-file\") pod \"limitador-limitador-64c94fd64f-chpjx\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:00.144512 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.144490 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10623230-b62a-40f8-828a-c3008738eed8-config-file\") pod \"limitador-limitador-64c94fd64f-chpjx\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:00.154877 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.154842 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jttxw\" (UniqueName: \"kubernetes.io/projected/10623230-b62a-40f8-828a-c3008738eed8-kube-api-access-jttxw\") pod \"limitador-limitador-64c94fd64f-chpjx\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:00.201055 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.201023 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:00.360318 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:00.360296 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-chpjx"] Jul 06 14:10:00.362603 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:10:00.362567 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10623230_b62a_40f8_828a_c3008738eed8.slice/crio-87feb82095dd2c2807ecc0e429a993ca96d33a787f12b8caf6c9dea41a66d319 WatchSource:0}: Error finding container 87feb82095dd2c2807ecc0e429a993ca96d33a787f12b8caf6c9dea41a66d319: Status 404 returned error can't find the container with id 87feb82095dd2c2807ecc0e429a993ca96d33a787f12b8caf6c9dea41a66d319 Jul 06 14:10:01.121519 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:01.121485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" event={"ID":"10623230-b62a-40f8-828a-c3008738eed8","Type":"ContainerStarted","Data":"87feb82095dd2c2807ecc0e429a993ca96d33a787f12b8caf6c9dea41a66d319"} Jul 06 14:10:03.133828 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:03.133792 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" event={"ID":"10623230-b62a-40f8-828a-c3008738eed8","Type":"ContainerStarted","Data":"be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725"} Jul 06 14:10:03.134309 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:03.133856 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:14.139847 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:14.139816 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:14.162302 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:14.162253 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" podStartSLOduration=12.488182945 podStartE2EDuration="15.162238122s" totalImagesPullingTime="2.674055177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:10:03.169347038 +0000 UTC m=+554.358396086" watchObservedRunningTime="2026-07-06 14:10:14.162238122 +0000 UTC m=+565.351287168" Jul 06 14:10:15.104692 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.104653 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-chpjx"] Jul 06 14:10:15.104976 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.104925 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" podUID="10623230-b62a-40f8-828a-c3008738eed8" containerName="limitador" containerID="cri-o://be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725" gracePeriod=30 Jul 06 14:10:15.649067 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.649035 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:15.696634 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.696547 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jttxw\" (UniqueName: \"kubernetes.io/projected/10623230-b62a-40f8-828a-c3008738eed8-kube-api-access-jttxw\") pod \"10623230-b62a-40f8-828a-c3008738eed8\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " Jul 06 14:10:15.696634 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.696605 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10623230-b62a-40f8-828a-c3008738eed8-config-file\") pod \"10623230-b62a-40f8-828a-c3008738eed8\" (UID: \"10623230-b62a-40f8-828a-c3008738eed8\") " Jul 06 14:10:15.697048 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.697026 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10623230-b62a-40f8-828a-c3008738eed8-config-file" (OuterVolumeSpecName: "config-file") pod "10623230-b62a-40f8-828a-c3008738eed8" (UID: "10623230-b62a-40f8-828a-c3008738eed8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:10:15.698699 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.698676 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10623230-b62a-40f8-828a-c3008738eed8-kube-api-access-jttxw" (OuterVolumeSpecName: "kube-api-access-jttxw") pod "10623230-b62a-40f8-828a-c3008738eed8" (UID: "10623230-b62a-40f8-828a-c3008738eed8"). InnerVolumeSpecName "kube-api-access-jttxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:10:15.798079 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.798043 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jttxw\" (UniqueName: \"kubernetes.io/projected/10623230-b62a-40f8-828a-c3008738eed8-kube-api-access-jttxw\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:10:15.798079 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:15.798075 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/10623230-b62a-40f8-828a-c3008738eed8-config-file\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:10:16.198932 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.198885 2562 generic.go:356] "Generic (PLEG): container finished" podID="10623230-b62a-40f8-828a-c3008738eed8" containerID="be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725" exitCode=0 Jul 06 14:10:16.199103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.198929 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" event={"ID":"10623230-b62a-40f8-828a-c3008738eed8","Type":"ContainerDied","Data":"be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725"} Jul 06 14:10:16.199103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.198987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" event={"ID":"10623230-b62a-40f8-828a-c3008738eed8","Type":"ContainerDied","Data":"87feb82095dd2c2807ecc0e429a993ca96d33a787f12b8caf6c9dea41a66d319"} Jul 06 14:10:16.199103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.199008 2562 scope.go:117] "RemoveContainer" containerID="be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725" Jul 06 14:10:16.199103 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.198943 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-chpjx" Jul 06 14:10:16.207822 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.207801 2562 scope.go:117] "RemoveContainer" containerID="be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725" Jul 06 14:10:16.208110 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:16.208090 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725\": container with ID starting with be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725 not found: ID does not exist" containerID="be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725" Jul 06 14:10:16.208157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.208121 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725"} err="failed to get container status \"be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725\": rpc error: code = NotFound desc = could not find container \"be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725\": container with ID starting with be222b8fae52e5c436fb6e555d9a4f1baba53a20ebabd9f6c8449ea045412725 not found: ID does not exist" Jul 06 14:10:16.224173 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.224146 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-chpjx"] Jul 06 14:10:16.230099 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:16.230074 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-chpjx"] Jul 06 14:10:17.390197 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:17.390163 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10623230-b62a-40f8-828a-c3008738eed8" path="/var/lib/kubelet/pods/10623230-b62a-40f8-828a-c3008738eed8/volumes" Jul 06 14:10:19.565017 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.564877 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-fqmhv"] Jul 06 14:10:19.565510 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.565485 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10623230-b62a-40f8-828a-c3008738eed8" containerName="limitador" Jul 06 14:10:19.565510 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.565509 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10623230-b62a-40f8-828a-c3008738eed8" containerName="limitador" Jul 06 14:10:19.565700 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.565629 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10623230-b62a-40f8-828a-c3008738eed8" containerName="limitador" Jul 06 14:10:19.568796 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.568766 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:19.571850 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.571827 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 14:10:19.572364 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.572342 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nk5cn\"" Jul 06 14:10:19.578982 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.578952 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-fqmhv"] Jul 06 14:10:19.634638 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.634601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhwbq\" (UniqueName: \"kubernetes.io/projected/ac7461fc-4773-484a-867c-0306504a0a9e-kube-api-access-vhwbq\") pod \"postgres-69fb4594cf-fqmhv\" (UID: \"ac7461fc-4773-484a-867c-0306504a0a9e\") " pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:19.634843 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.634753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ac7461fc-4773-484a-867c-0306504a0a9e-data\") pod \"postgres-69fb4594cf-fqmhv\" (UID: \"ac7461fc-4773-484a-867c-0306504a0a9e\") " pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:19.735819 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.735780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ac7461fc-4773-484a-867c-0306504a0a9e-data\") pod \"postgres-69fb4594cf-fqmhv\" (UID: \"ac7461fc-4773-484a-867c-0306504a0a9e\") " pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:19.736032 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.735844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhwbq\" (UniqueName: \"kubernetes.io/projected/ac7461fc-4773-484a-867c-0306504a0a9e-kube-api-access-vhwbq\") pod \"postgres-69fb4594cf-fqmhv\" (UID: \"ac7461fc-4773-484a-867c-0306504a0a9e\") " pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:19.736175 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.736155 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ac7461fc-4773-484a-867c-0306504a0a9e-data\") pod \"postgres-69fb4594cf-fqmhv\" (UID: \"ac7461fc-4773-484a-867c-0306504a0a9e\") " pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:19.747456 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.747423 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhwbq\" (UniqueName: \"kubernetes.io/projected/ac7461fc-4773-484a-867c-0306504a0a9e-kube-api-access-vhwbq\") pod \"postgres-69fb4594cf-fqmhv\" (UID: \"ac7461fc-4773-484a-867c-0306504a0a9e\") " pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:19.881437 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:19.881363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:20.035230 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:20.035201 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-fqmhv"] Jul 06 14:10:20.037515 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:10:20.037484 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7461fc_4773_484a_867c_0306504a0a9e.slice/crio-395630c3d63f95ab154f1396ae277e8587a60e3268bf678ed3182ced678b1045 WatchSource:0}: Error finding container 395630c3d63f95ab154f1396ae277e8587a60e3268bf678ed3182ced678b1045: Status 404 returned error can't find the container with id 395630c3d63f95ab154f1396ae277e8587a60e3268bf678ed3182ced678b1045 Jul 06 14:10:20.220491 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:20.220402 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-fqmhv" event={"ID":"ac7461fc-4773-484a-867c-0306504a0a9e","Type":"ContainerStarted","Data":"395630c3d63f95ab154f1396ae277e8587a60e3268bf678ed3182ced678b1045"} Jul 06 14:10:25.252074 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:25.252034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-fqmhv" event={"ID":"ac7461fc-4773-484a-867c-0306504a0a9e","Type":"ContainerStarted","Data":"4f9049ea9a6b21bd5de96f4284a3ba2be71f440d43f151511bb02dd6781d0815"} Jul 06 14:10:25.252513 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:25.252104 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:25.272788 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:25.272732 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-fqmhv" podStartSLOduration=1.6617407050000002 podStartE2EDuration="6.272716958s" totalImagesPullingTime="4.610976253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:10:25.270363369 +0000 UTC m=+576.459412415" watchObservedRunningTime="2026-07-06 14:10:25.272716958 +0000 UTC m=+576.461766005" Jul 06 14:10:31.287845 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:31.287816 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-fqmhv" Jul 06 14:10:31.892683 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:31.892650 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9bfbv"] Jul 06 14:10:31.898112 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:31.898082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" Jul 06 14:10:31.901081 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:31.901053 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4wt6r\"" Jul 06 14:10:31.906034 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:31.906002 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9bfbv"] Jul 06 14:10:31.958533 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:31.958499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fm5g\" (UniqueName: \"kubernetes.io/projected/6e718d54-6f78-401b-9594-2f6013892c7b-kube-api-access-9fm5g\") pod \"authorino-5d4b67c446-9bfbv\" (UID: \"6e718d54-6f78-401b-9594-2f6013892c7b\") " pod="kuadrant-system/authorino-5d4b67c446-9bfbv" Jul 06 14:10:32.059691 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:32.059653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fm5g\" (UniqueName: \"kubernetes.io/projected/6e718d54-6f78-401b-9594-2f6013892c7b-kube-api-access-9fm5g\") pod \"authorino-5d4b67c446-9bfbv\" (UID: \"6e718d54-6f78-401b-9594-2f6013892c7b\") " pod="kuadrant-system/authorino-5d4b67c446-9bfbv" Jul 06 14:10:32.073172 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:32.073144 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fm5g\" (UniqueName: \"kubernetes.io/projected/6e718d54-6f78-401b-9594-2f6013892c7b-kube-api-access-9fm5g\") pod \"authorino-5d4b67c446-9bfbv\" (UID: \"6e718d54-6f78-401b-9594-2f6013892c7b\") " pod="kuadrant-system/authorino-5d4b67c446-9bfbv" Jul 06 14:10:32.151142 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:32.151036 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9bfbv"] Jul 06 14:10:32.151411 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:32.151392 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" Jul 06 14:10:32.290639 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:32.290553 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9bfbv"] Jul 06 14:10:32.293217 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:10:32.293189 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e718d54_6f78_401b_9594_2f6013892c7b.slice/crio-a62684cf1f0b122e63bb5f875277cba5e829a18469fe7c8dab32b27ec3decfb8 WatchSource:0}: Error finding container a62684cf1f0b122e63bb5f875277cba5e829a18469fe7c8dab32b27ec3decfb8: Status 404 returned error can't find the container with id a62684cf1f0b122e63bb5f875277cba5e829a18469fe7c8dab32b27ec3decfb8 Jul 06 14:10:33.301390 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:33.301351 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" event={"ID":"6e718d54-6f78-401b-9594-2f6013892c7b","Type":"ContainerStarted","Data":"a62684cf1f0b122e63bb5f875277cba5e829a18469fe7c8dab32b27ec3decfb8"} Jul 06 14:10:37.326273 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:37.326235 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" event={"ID":"6e718d54-6f78-401b-9594-2f6013892c7b","Type":"ContainerStarted","Data":"93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78"} Jul 06 14:10:37.326729 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:37.326322 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" podUID="6e718d54-6f78-401b-9594-2f6013892c7b" containerName="authorino" containerID="cri-o://93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78" gracePeriod=30 Jul 06 14:10:37.354606 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:37.354551 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" podStartSLOduration=2.121199706 podStartE2EDuration="6.35453343s" totalImagesPullingTime="4.233333724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:10:37.353965364 +0000 UTC m=+588.543014412" watchObservedRunningTime="2026-07-06 14:10:37.35453343 +0000 UTC m=+588.543582477" Jul 06 14:10:37.584221 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:37.584157 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" Jul 06 14:10:37.722856 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:37.722818 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fm5g\" (UniqueName: \"kubernetes.io/projected/6e718d54-6f78-401b-9594-2f6013892c7b-kube-api-access-9fm5g\") pod \"6e718d54-6f78-401b-9594-2f6013892c7b\" (UID: \"6e718d54-6f78-401b-9594-2f6013892c7b\") " Jul 06 14:10:37.724826 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:37.724788 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e718d54-6f78-401b-9594-2f6013892c7b-kube-api-access-9fm5g" (OuterVolumeSpecName: "kube-api-access-9fm5g") pod "6e718d54-6f78-401b-9594-2f6013892c7b" (UID: "6e718d54-6f78-401b-9594-2f6013892c7b"). InnerVolumeSpecName "kube-api-access-9fm5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:10:37.824195 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:37.824143 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fm5g\" (UniqueName: \"kubernetes.io/projected/6e718d54-6f78-401b-9594-2f6013892c7b-kube-api-access-9fm5g\") on node \"ip-10-0-129-92.ec2.internal\" DevicePath \"\"" Jul 06 14:10:38.332057 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.332016 2562 generic.go:356] "Generic (PLEG): container finished" podID="6e718d54-6f78-401b-9594-2f6013892c7b" containerID="93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78" exitCode=0 Jul 06 14:10:38.332542 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.332061 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" Jul 06 14:10:38.332542 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.332075 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" event={"ID":"6e718d54-6f78-401b-9594-2f6013892c7b","Type":"ContainerDied","Data":"93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78"} Jul 06 14:10:38.332542 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.332110 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-9bfbv" event={"ID":"6e718d54-6f78-401b-9594-2f6013892c7b","Type":"ContainerDied","Data":"a62684cf1f0b122e63bb5f875277cba5e829a18469fe7c8dab32b27ec3decfb8"} Jul 06 14:10:38.332542 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.332129 2562 scope.go:117] "RemoveContainer" containerID="93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78" Jul 06 14:10:38.341028 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.341007 2562 scope.go:117] "RemoveContainer" containerID="93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78" Jul 06 14:10:38.341284 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:38.341267 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78\": container with ID starting with 93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78 not found: ID does not exist" containerID="93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78" Jul 06 14:10:38.341330 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.341291 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78"} err="failed to get container status \"93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78\": rpc error: code = NotFound desc = could not find container \"93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78\": container with ID starting with 93c2d13f95e6b7fdf3b9043ffb27275d7eaba341b5b44c0dd902d9c33323fc78 not found: ID does not exist" Jul 06 14:10:38.360226 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.360198 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9bfbv"] Jul 06 14:10:38.363481 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.363456 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-9bfbv"] Jul 06 14:10:38.703231 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.703136 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f748f5f45-9r5lk"] Jul 06 14:10:38.703753 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.703727 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e718d54-6f78-401b-9594-2f6013892c7b" containerName="authorino" Jul 06 14:10:38.703888 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.703777 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e718d54-6f78-401b-9594-2f6013892c7b" containerName="authorino" Jul 06 14:10:38.703984 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.703927 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e718d54-6f78-401b-9594-2f6013892c7b" containerName="authorino" Jul 06 14:10:38.707281 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.707254 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:38.710152 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.710127 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tkgrg\"" Jul 06 14:10:38.710152 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.710150 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 14:10:38.710338 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.710198 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 14:10:38.722522 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.722496 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f748f5f45-9r5lk"] Jul 06 14:10:38.834483 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.834446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc3d1864-d649-4722-a99a-da0adc569d33-webhook-cert\") pod \"maas-controller-7f748f5f45-9r5lk\" (UID: \"cc3d1864-d649-4722-a99a-da0adc569d33\") " pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:38.834668 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.834588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztfw7\" (UniqueName: \"kubernetes.io/projected/cc3d1864-d649-4722-a99a-da0adc569d33-kube-api-access-ztfw7\") pod \"maas-controller-7f748f5f45-9r5lk\" (UID: \"cc3d1864-d649-4722-a99a-da0adc569d33\") " pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:38.927723 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.927683 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-795f8494b4-dcsk9"] Jul 06 14:10:38.930661 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.930638 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:38.935070 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.935049 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztfw7\" (UniqueName: \"kubernetes.io/projected/cc3d1864-d649-4722-a99a-da0adc569d33-kube-api-access-ztfw7\") pod \"maas-controller-7f748f5f45-9r5lk\" (UID: \"cc3d1864-d649-4722-a99a-da0adc569d33\") " pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:38.935186 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.935100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc3d1864-d649-4722-a99a-da0adc569d33-webhook-cert\") pod \"maas-controller-7f748f5f45-9r5lk\" (UID: \"cc3d1864-d649-4722-a99a-da0adc569d33\") " pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:38.937677 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.937652 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc3d1864-d649-4722-a99a-da0adc569d33-webhook-cert\") pod \"maas-controller-7f748f5f45-9r5lk\" (UID: \"cc3d1864-d649-4722-a99a-da0adc569d33\") " pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:38.945671 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.945645 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztfw7\" (UniqueName: \"kubernetes.io/projected/cc3d1864-d649-4722-a99a-da0adc569d33-kube-api-access-ztfw7\") pod \"maas-controller-7f748f5f45-9r5lk\" (UID: \"cc3d1864-d649-4722-a99a-da0adc569d33\") " pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:38.963046 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:38.960475 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795f8494b4-dcsk9"] Jul 06 14:10:39.018200 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.018152 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:39.036061 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.036029 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn2b5\" (UniqueName: \"kubernetes.io/projected/67ad67f1-49f1-4303-a827-a0c7cd898805-kube-api-access-wn2b5\") pod \"maas-controller-795f8494b4-dcsk9\" (UID: \"67ad67f1-49f1-4303-a827-a0c7cd898805\") " pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:39.036273 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.036244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67ad67f1-49f1-4303-a827-a0c7cd898805-webhook-cert\") pod \"maas-controller-795f8494b4-dcsk9\" (UID: \"67ad67f1-49f1-4303-a827-a0c7cd898805\") " pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:39.137662 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.137625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67ad67f1-49f1-4303-a827-a0c7cd898805-webhook-cert\") pod \"maas-controller-795f8494b4-dcsk9\" (UID: \"67ad67f1-49f1-4303-a827-a0c7cd898805\") " pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:39.137825 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.137699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn2b5\" (UniqueName: \"kubernetes.io/projected/67ad67f1-49f1-4303-a827-a0c7cd898805-kube-api-access-wn2b5\") pod \"maas-controller-795f8494b4-dcsk9\" (UID: \"67ad67f1-49f1-4303-a827-a0c7cd898805\") " pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:39.140283 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.140251 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67ad67f1-49f1-4303-a827-a0c7cd898805-webhook-cert\") pod \"maas-controller-795f8494b4-dcsk9\" (UID: \"67ad67f1-49f1-4303-a827-a0c7cd898805\") " pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:39.152995 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.152965 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f748f5f45-9r5lk"] Jul 06 14:10:39.153573 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.153547 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn2b5\" (UniqueName: \"kubernetes.io/projected/67ad67f1-49f1-4303-a827-a0c7cd898805-kube-api-access-wn2b5\") pod \"maas-controller-795f8494b4-dcsk9\" (UID: \"67ad67f1-49f1-4303-a827-a0c7cd898805\") " pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:39.155586 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:10:39.155561 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3d1864_d649_4722_a99a_da0adc569d33.slice/crio-133e0d1ff624403dc162b27cbca26330a88df3c5921bbc17d44421b63fd10fe2 WatchSource:0}: Error finding container 133e0d1ff624403dc162b27cbca26330a88df3c5921bbc17d44421b63fd10fe2: Status 404 returned error can't find the container with id 133e0d1ff624403dc162b27cbca26330a88df3c5921bbc17d44421b63fd10fe2 Jul 06 14:10:39.256167 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.256066 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:39.341438 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.341400 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" event={"ID":"cc3d1864-d649-4722-a99a-da0adc569d33","Type":"ContainerStarted","Data":"133e0d1ff624403dc162b27cbca26330a88df3c5921bbc17d44421b63fd10fe2"} Jul 06 14:10:39.389697 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.389665 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e718d54-6f78-401b-9594-2f6013892c7b" path="/var/lib/kubelet/pods/6e718d54-6f78-401b-9594-2f6013892c7b/volumes" Jul 06 14:10:39.396462 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:39.396440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-795f8494b4-dcsk9"] Jul 06 14:10:39.398496 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:10:39.398474 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ad67f1_49f1_4303_a827_a0c7cd898805.slice/crio-2a6e9e0cab3112c5d405c37e9053f8aa027bef035b6e8903632202f263e05f8a WatchSource:0}: Error finding container 2a6e9e0cab3112c5d405c37e9053f8aa027bef035b6e8903632202f263e05f8a: Status 404 returned error can't find the container with id 2a6e9e0cab3112c5d405c37e9053f8aa027bef035b6e8903632202f263e05f8a Jul 06 14:10:40.353654 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:40.353569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795f8494b4-dcsk9" event={"ID":"67ad67f1-49f1-4303-a827-a0c7cd898805","Type":"ContainerStarted","Data":"2a6e9e0cab3112c5d405c37e9053f8aa027bef035b6e8903632202f263e05f8a"} Jul 06 14:10:42.368779 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:42.368742 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc3d1864-d649-4722-a99a-da0adc569d33" containerID="45e0eb09192230d52212e2875b2ceb022c8bbee0c1cf5e40ac01110eb1c4758f" exitCode=2 Jul 06 14:10:42.369275 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:42.368839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" event={"ID":"cc3d1864-d649-4722-a99a-da0adc569d33","Type":"ContainerDied","Data":"45e0eb09192230d52212e2875b2ceb022c8bbee0c1cf5e40ac01110eb1c4758f"} Jul 06 14:10:42.369275 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:42.369108 2562 scope.go:117] "RemoveContainer" containerID="45e0eb09192230d52212e2875b2ceb022c8bbee0c1cf5e40ac01110eb1c4758f" Jul 06 14:10:42.370623 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:42.370598 2562 generic.go:356] "Generic (PLEG): container finished" podID="67ad67f1-49f1-4303-a827-a0c7cd898805" containerID="23e8c8485d67998506264bce2a34f403fd5ae3703877b957b30bdb908b34298e" exitCode=2 Jul 06 14:10:42.370746 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:42.370639 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795f8494b4-dcsk9" event={"ID":"67ad67f1-49f1-4303-a827-a0c7cd898805","Type":"ContainerDied","Data":"23e8c8485d67998506264bce2a34f403fd5ae3703877b957b30bdb908b34298e"} Jul 06 14:10:42.370851 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:42.370840 2562 scope.go:117] "RemoveContainer" containerID="23e8c8485d67998506264bce2a34f403fd5ae3703877b957b30bdb908b34298e" Jul 06 14:10:43.377620 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.377586 2562 generic.go:356] "Generic (PLEG): container finished" podID="67ad67f1-49f1-4303-a827-a0c7cd898805" containerID="6db04edbd4fbb38871fd15a15e81c485f3571cb0ca3f5da767f41193babcf62c" exitCode=2 Jul 06 14:10:43.378090 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.377620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795f8494b4-dcsk9" event={"ID":"67ad67f1-49f1-4303-a827-a0c7cd898805","Type":"ContainerDied","Data":"6db04edbd4fbb38871fd15a15e81c485f3571cb0ca3f5da767f41193babcf62c"} Jul 06 14:10:43.378090 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.377684 2562 scope.go:117] "RemoveContainer" containerID="23e8c8485d67998506264bce2a34f403fd5ae3703877b957b30bdb908b34298e" Jul 06 14:10:43.378090 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.377876 2562 scope.go:117] "RemoveContainer" containerID="6db04edbd4fbb38871fd15a15e81c485f3571cb0ca3f5da767f41193babcf62c" Jul 06 14:10:43.378231 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:43.378184 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:10:43.379613 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.379592 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc3d1864-d649-4722-a99a-da0adc569d33" containerID="7c34f4338b58ba94d193bfaced4e4d538c941db98c3f8185604b1377ba56826b" exitCode=2 Jul 06 14:10:43.379705 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.379642 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" event={"ID":"cc3d1864-d649-4722-a99a-da0adc569d33","Type":"ContainerDied","Data":"7c34f4338b58ba94d193bfaced4e4d538c941db98c3f8185604b1377ba56826b"} Jul 06 14:10:43.379876 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.379863 2562 scope.go:117] "RemoveContainer" containerID="7c34f4338b58ba94d193bfaced4e4d538c941db98c3f8185604b1377ba56826b" Jul 06 14:10:43.380068 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:43.380052 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:10:43.395028 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:43.395013 2562 scope.go:117] "RemoveContainer" containerID="45e0eb09192230d52212e2875b2ceb022c8bbee0c1cf5e40ac01110eb1c4758f" Jul 06 14:10:44.386285 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:44.386254 2562 scope.go:117] "RemoveContainer" containerID="7c34f4338b58ba94d193bfaced4e4d538c941db98c3f8185604b1377ba56826b" Jul 06 14:10:44.386699 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:44.386451 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:10:44.387815 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:44.387795 2562 scope.go:117] "RemoveContainer" containerID="6db04edbd4fbb38871fd15a15e81c485f3571cb0ca3f5da767f41193babcf62c" Jul 06 14:10:44.387983 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:44.387963 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:10:49.019197 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:49.019157 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:49.019614 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:49.019582 2562 scope.go:117] "RemoveContainer" containerID="7c34f4338b58ba94d193bfaced4e4d538c941db98c3f8185604b1377ba56826b" Jul 06 14:10:49.019797 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:49.019772 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:10:49.256177 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:49.256150 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:49.256505 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:49.256489 2562 scope.go:117] "RemoveContainer" containerID="6db04edbd4fbb38871fd15a15e81c485f3571cb0ca3f5da767f41193babcf62c" Jul 06 14:10:49.256814 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:49.256776 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:10:49.275735 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:49.275653 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:10:49.276069 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:49.276050 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vcfvm_aa080866-0ad2-43b4-90b6-68339fdcffc2/ovn-acl-logging/0.log" Jul 06 14:10:59.019150 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.019119 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:10:59.019572 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.019474 2562 scope.go:117] "RemoveContainer" containerID="7c34f4338b58ba94d193bfaced4e4d538c941db98c3f8185604b1377ba56826b" Jul 06 14:10:59.256438 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.256405 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:10:59.256806 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.256791 2562 scope.go:117] "RemoveContainer" containerID="6db04edbd4fbb38871fd15a15e81c485f3571cb0ca3f5da767f41193babcf62c" Jul 06 14:10:59.466201 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.466169 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc3d1864-d649-4722-a99a-da0adc569d33" containerID="95c21f23875cced2e307d8f72008c09c0767eb8b1709d7a25085757104372d8e" exitCode=2 Jul 06 14:10:59.466364 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.466240 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" event={"ID":"cc3d1864-d649-4722-a99a-da0adc569d33","Type":"ContainerDied","Data":"95c21f23875cced2e307d8f72008c09c0767eb8b1709d7a25085757104372d8e"} Jul 06 14:10:59.466364 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.466306 2562 scope.go:117] "RemoveContainer" containerID="7c34f4338b58ba94d193bfaced4e4d538c941db98c3f8185604b1377ba56826b" Jul 06 14:10:59.466631 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:10:59.466610 2562 scope.go:117] "RemoveContainer" containerID="95c21f23875cced2e307d8f72008c09c0767eb8b1709d7a25085757104372d8e" Jul 06 14:10:59.466847 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:10:59.466824 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:11:00.473449 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:00.473413 2562 generic.go:356] "Generic (PLEG): container finished" podID="67ad67f1-49f1-4303-a827-a0c7cd898805" containerID="51146c829a535385c04352e2696cea3647eff410797bbafa8a5489c1e9ccecce" exitCode=2 Jul 06 14:11:00.473982 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:00.473485 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795f8494b4-dcsk9" event={"ID":"67ad67f1-49f1-4303-a827-a0c7cd898805","Type":"ContainerDied","Data":"51146c829a535385c04352e2696cea3647eff410797bbafa8a5489c1e9ccecce"} Jul 06 14:11:00.473982 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:00.473542 2562 scope.go:117] "RemoveContainer" containerID="6db04edbd4fbb38871fd15a15e81c485f3571cb0ca3f5da767f41193babcf62c" Jul 06 14:11:00.473982 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:00.473882 2562 scope.go:117] "RemoveContainer" containerID="51146c829a535385c04352e2696cea3647eff410797bbafa8a5489c1e9ccecce" Jul 06 14:11:00.474196 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:00.474177 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:11:09.018935 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:09.018856 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:11:09.019478 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:09.019348 2562 scope.go:117] "RemoveContainer" containerID="95c21f23875cced2e307d8f72008c09c0767eb8b1709d7a25085757104372d8e" Jul 06 14:11:09.019600 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:09.019576 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:11:09.256719 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:09.256672 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:11:09.257112 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:09.257096 2562 scope.go:117] "RemoveContainer" containerID="51146c829a535385c04352e2696cea3647eff410797bbafa8a5489c1e9ccecce" Jul 06 14:11:09.257273 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:09.257255 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:11:19.018413 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:19.018329 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:11:19.018797 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:19.018733 2562 scope.go:117] "RemoveContainer" containerID="95c21f23875cced2e307d8f72008c09c0767eb8b1709d7a25085757104372d8e" Jul 06 14:11:19.018997 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:19.018970 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:11:19.257156 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:19.257120 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:11:19.257513 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:19.257500 2562 scope.go:117] "RemoveContainer" containerID="51146c829a535385c04352e2696cea3647eff410797bbafa8a5489c1e9ccecce" Jul 06 14:11:19.257703 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:19.257683 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:11:29.385380 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:29.385354 2562 scope.go:117] "RemoveContainer" containerID="51146c829a535385c04352e2696cea3647eff410797bbafa8a5489c1e9ccecce" Jul 06 14:11:30.633332 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:30.633294 2562 generic.go:356] "Generic (PLEG): container finished" podID="67ad67f1-49f1-4303-a827-a0c7cd898805" containerID="2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d" exitCode=2 Jul 06 14:11:30.633815 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:30.633368 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795f8494b4-dcsk9" event={"ID":"67ad67f1-49f1-4303-a827-a0c7cd898805","Type":"ContainerDied","Data":"2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d"} Jul 06 14:11:30.633815 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:30.633426 2562 scope.go:117] "RemoveContainer" containerID="51146c829a535385c04352e2696cea3647eff410797bbafa8a5489c1e9ccecce" Jul 06 14:11:30.633815 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:30.633812 2562 scope.go:117] "RemoveContainer" containerID="2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d" Jul 06 14:11:30.634111 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:30.634089 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:11:33.382972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:33.382939 2562 scope.go:117] "RemoveContainer" containerID="95c21f23875cced2e307d8f72008c09c0767eb8b1709d7a25085757104372d8e" Jul 06 14:11:33.383416 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:33.383363 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:11:34.658021 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:34.657984 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc3d1864-d649-4722-a99a-da0adc569d33" containerID="04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3" exitCode=2 Jul 06 14:11:34.658441 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:34.658059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" event={"ID":"cc3d1864-d649-4722-a99a-da0adc569d33","Type":"ContainerDied","Data":"04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3"} Jul 06 14:11:34.658441 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:34.658163 2562 scope.go:117] "RemoveContainer" containerID="95c21f23875cced2e307d8f72008c09c0767eb8b1709d7a25085757104372d8e" Jul 06 14:11:34.658543 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:34.658479 2562 scope.go:117] "RemoveContainer" containerID="04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3" Jul 06 14:11:34.658702 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:34.658683 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:11:39.018644 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:39.018606 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:11:39.019109 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:39.018671 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:11:39.019109 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:39.019024 2562 scope.go:117] "RemoveContainer" containerID="04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3" Jul 06 14:11:39.019265 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:39.019243 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:11:39.257204 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:39.257165 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:11:39.257384 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:39.257256 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:11:39.257627 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:39.257602 2562 scope.go:117] "RemoveContainer" containerID="2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d" Jul 06 14:11:39.257809 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:39.257789 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:11:51.382972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:51.382940 2562 scope.go:117] "RemoveContainer" containerID="04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3" Jul 06 14:11:51.383435 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:51.383159 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:11:53.383211 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:11:53.383177 2562 scope.go:117] "RemoveContainer" containerID="2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d" Jul 06 14:11:53.383598 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:11:53.383343 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:12:02.384112 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:02.384074 2562 scope.go:117] "RemoveContainer" containerID="04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3" Jul 06 14:12:02.384687 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:02.384267 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:12:04.383392 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:04.383357 2562 scope.go:117] "RemoveContainer" containerID="2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d" Jul 06 14:12:04.383806 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:04.383585 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:12:14.383311 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:14.383281 2562 scope.go:117] "RemoveContainer" containerID="04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3" Jul 06 14:12:15.870565 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:15.870531 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc3d1864-d649-4722-a99a-da0adc569d33" containerID="db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8" exitCode=2 Jul 06 14:12:15.871015 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:15.870602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" event={"ID":"cc3d1864-d649-4722-a99a-da0adc569d33","Type":"ContainerDied","Data":"db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8"} Jul 06 14:12:15.871015 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:15.870661 2562 scope.go:117] "RemoveContainer" containerID="04cace9413971a4827c2dda898c92c6f1a0dcffdb8cf74112a16f4aea17d7be3" Jul 06 14:12:15.871015 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:15.870971 2562 scope.go:117] "RemoveContainer" containerID="db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8" Jul 06 14:12:15.871198 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:15.871172 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:12:16.383295 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:16.383262 2562 scope.go:117] "RemoveContainer" containerID="2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d" Jul 06 14:12:16.878848 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:16.878815 2562 generic.go:356] "Generic (PLEG): container finished" podID="67ad67f1-49f1-4303-a827-a0c7cd898805" containerID="a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37" exitCode=2 Jul 06 14:12:16.879296 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:16.878887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-795f8494b4-dcsk9" event={"ID":"67ad67f1-49f1-4303-a827-a0c7cd898805","Type":"ContainerDied","Data":"a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37"} Jul 06 14:12:16.879296 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:16.878973 2562 scope.go:117] "RemoveContainer" containerID="2f2583c1622d483cef0afec70c78cffa59c436f130c9d6f8cf9be36d67c58d9d" Jul 06 14:12:16.879399 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:16.879301 2562 scope.go:117] "RemoveContainer" containerID="a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37" Jul 06 14:12:16.879561 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:16.879543 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:12:19.018539 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:19.018498 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:12:19.018983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:19.018558 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" Jul 06 14:12:19.018983 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:19.018876 2562 scope.go:117] "RemoveContainer" containerID="db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8" Jul 06 14:12:19.019127 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:19.019100 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:12:19.256950 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:19.256891 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:12:19.257141 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:19.256973 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-795f8494b4-dcsk9" Jul 06 14:12:19.257307 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:19.257293 2562 scope.go:117] "RemoveContainer" containerID="a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37" Jul 06 14:12:19.257545 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:19.257523 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:12:33.383072 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:33.383035 2562 scope.go:117] "RemoveContainer" containerID="a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37" Jul 06 14:12:33.383625 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:33.383267 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:12:34.383233 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:34.383191 2562 scope.go:117] "RemoveContainer" containerID="db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8" Jul 06 14:12:34.383705 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:34.383459 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:12:42.491659 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:42.491586 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sbz6m_e52cb6c2-5732-4193-9f0b-7277f55177fd/manager/0.log" Jul 06 14:12:42.605216 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:42.605187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d8vl9_5693d981-477d-458d-a525-938e6f8724f9/manager/0.log" Jul 06 14:12:42.713319 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:42.713279 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-795f8494b4-dcsk9_67ad67f1-49f1-4303-a827-a0c7cd898805/manager/4.log" Jul 06 14:12:42.827147 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:42.827118 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f748f5f45-9r5lk_cc3d1864-d649-4722-a99a-da0adc569d33/manager/4.log" Jul 06 14:12:43.178410 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:43.178334 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d88dcf597-kjrrq_bc80f37c-c88a-4e80-9f07-a321b1e8bfb1/copy-manifests/0.log" Jul 06 14:12:43.191180 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:43.191138 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d88dcf597-kjrrq_bc80f37c-c88a-4e80-9f07-a321b1e8bfb1/manager/0.log" Jul 06 14:12:43.552497 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:43.552471 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-fqmhv_ac7461fc-4773-484a-867c-0306504a0a9e/postgres/0.log" Jul 06 14:12:45.139607 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:45.139579 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jdscf_ef4b7ce5-b65d-4381-a6ab-89dc62e908f8/registry-server/0.log" Jul 06 14:12:45.383192 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:45.383163 2562 scope.go:117] "RemoveContainer" containerID="a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37" Jul 06 14:12:45.383385 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:45.383363 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:12:45.481696 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:45.481602 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ldld2_69dcbb7c-3c67-43dc-827b-1869518d3c26/manager/0.log" Jul 06 14:12:45.828114 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:45.828083 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gbst7_42f86e98-10ae-4a13-9639-c367f911d3d8/istio-proxy/0.log" Jul 06 14:12:46.274161 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:46.274136 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-nmqnb_8f349803-9b28-4683-b81f-9bb0f4b0639a/istio-proxy/0.log" Jul 06 14:12:49.385824 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:49.385787 2562 scope.go:117] "RemoveContainer" containerID="db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8" Jul 06 14:12:49.386244 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:12:49.386112 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:12:50.435339 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.435296 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lzps2/must-gather-njqwb"] Jul 06 14:12:50.439053 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.439031 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.441816 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.441796 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lzps2\"/\"kube-root-ca.crt\"" Jul 06 14:12:50.443125 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.443106 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lzps2\"/\"default-dockercfg-qj6w5\"" Jul 06 14:12:50.443228 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.443113 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lzps2\"/\"openshift-service-ca.crt\"" Jul 06 14:12:50.448951 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.448925 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lzps2/must-gather-njqwb"] Jul 06 14:12:50.548839 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.548798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcccm\" (UniqueName: \"kubernetes.io/projected/ff16b6d5-0071-4dad-809c-b565c315f4a9-kube-api-access-kcccm\") pod \"must-gather-njqwb\" (UID: \"ff16b6d5-0071-4dad-809c-b565c315f4a9\") " pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.549036 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.548920 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ff16b6d5-0071-4dad-809c-b565c315f4a9-must-gather-output\") pod \"must-gather-njqwb\" (UID: \"ff16b6d5-0071-4dad-809c-b565c315f4a9\") " pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.650425 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.650387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ff16b6d5-0071-4dad-809c-b565c315f4a9-must-gather-output\") pod \"must-gather-njqwb\" (UID: \"ff16b6d5-0071-4dad-809c-b565c315f4a9\") " pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.650600 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.650556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcccm\" (UniqueName: \"kubernetes.io/projected/ff16b6d5-0071-4dad-809c-b565c315f4a9-kube-api-access-kcccm\") pod \"must-gather-njqwb\" (UID: \"ff16b6d5-0071-4dad-809c-b565c315f4a9\") " pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.650722 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.650701 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ff16b6d5-0071-4dad-809c-b565c315f4a9-must-gather-output\") pod \"must-gather-njqwb\" (UID: \"ff16b6d5-0071-4dad-809c-b565c315f4a9\") " pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.660295 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.660267 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcccm\" (UniqueName: \"kubernetes.io/projected/ff16b6d5-0071-4dad-809c-b565c315f4a9-kube-api-access-kcccm\") pod \"must-gather-njqwb\" (UID: \"ff16b6d5-0071-4dad-809c-b565c315f4a9\") " pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.749441 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.749355 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lzps2/must-gather-njqwb" Jul 06 14:12:50.884656 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:50.884628 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lzps2/must-gather-njqwb"] Jul 06 14:12:50.886357 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:12:50.886328 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff16b6d5_0071_4dad_809c_b565c315f4a9.slice/crio-1cc5b44b7143fc4d7ea49402eb0dfdb137f1aa9c3aa33911b6f3b52fa477a2b9 WatchSource:0}: Error finding container 1cc5b44b7143fc4d7ea49402eb0dfdb137f1aa9c3aa33911b6f3b52fa477a2b9: Status 404 returned error can't find the container with id 1cc5b44b7143fc4d7ea49402eb0dfdb137f1aa9c3aa33911b6f3b52fa477a2b9 Jul 06 14:12:51.053439 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:51.053401 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lzps2/must-gather-njqwb" event={"ID":"ff16b6d5-0071-4dad-809c-b565c315f4a9","Type":"ContainerStarted","Data":"1cc5b44b7143fc4d7ea49402eb0dfdb137f1aa9c3aa33911b6f3b52fa477a2b9"} Jul 06 14:12:53.068086 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:53.067148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lzps2/must-gather-njqwb" event={"ID":"ff16b6d5-0071-4dad-809c-b565c315f4a9","Type":"ContainerStarted","Data":"638e2b9d489df0f782f7ce27d6b547f19d4126f109ddf529df3943b2052eb390"} Jul 06 14:12:53.068086 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:53.067208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lzps2/must-gather-njqwb" event={"ID":"ff16b6d5-0071-4dad-809c-b565c315f4a9","Type":"ContainerStarted","Data":"ae735a2cb877d656114eae351f17d047820d66e184147d91a89d0c53f2866a61"} Jul 06 14:12:53.088314 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:53.088260 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lzps2/must-gather-njqwb" podStartSLOduration=1.8231908300000002 podStartE2EDuration="3.088242028s" totalImagesPullingTime="1.265051198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:12:53.08612856 +0000 UTC m=+724.275177607" watchObservedRunningTime="2026-07-06 14:12:53.088242028 +0000 UTC m=+724.277291075" Jul 06 14:12:54.087141 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:54.087105 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9g6dg_f31c9785-76aa-4397-8383-3d5d06adc6b8/global-pull-secret-syncer/0.log" Jul 06 14:12:54.263341 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:54.263307 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ffc78_b32a7a04-eee4-4d9d-8026-97d8501d6d50/konnectivity-agent/0.log" Jul 06 14:12:54.320892 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:54.320862 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-92.ec2.internal_1524ac3c95d4a1a172ad30214b4048df/haproxy/0.log" Jul 06 14:12:59.173637 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:59.173337 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-jdscf_ef4b7ce5-b65d-4381-a6ab-89dc62e908f8/registry-server/0.log" Jul 06 14:12:59.366208 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:12:59.366171 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ldld2_69dcbb7c-3c67-43dc-827b-1869518d3c26/manager/0.log" Jul 06 14:13:00.383417 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:00.383381 2562 scope.go:117] "RemoveContainer" containerID="a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37" Jul 06 14:13:00.384007 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:13:00.383679 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:13:00.956917 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:00.956873 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d548fc36-d7cd-4dd7-9b4d-5f48a58624da/alertmanager/0.log" Jul 06 14:13:01.002011 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.001982 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d548fc36-d7cd-4dd7-9b4d-5f48a58624da/config-reloader/0.log" Jul 06 14:13:01.028889 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.028861 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d548fc36-d7cd-4dd7-9b4d-5f48a58624da/kube-rbac-proxy-web/0.log" Jul 06 14:13:01.055123 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.055084 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d548fc36-d7cd-4dd7-9b4d-5f48a58624da/kube-rbac-proxy/0.log" Jul 06 14:13:01.083932 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.083860 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d548fc36-d7cd-4dd7-9b4d-5f48a58624da/kube-rbac-proxy-metric/0.log" Jul 06 14:13:01.111247 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.111214 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d548fc36-d7cd-4dd7-9b4d-5f48a58624da/prom-label-proxy/0.log" Jul 06 14:13:01.138400 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.138376 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d548fc36-d7cd-4dd7-9b4d-5f48a58624da/init-config-reloader/0.log" Jul 06 14:13:01.217066 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.216932 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-9d6vz_dc19b82f-3557-445e-bcd7-33766be1a5cc/kube-state-metrics/0.log" Jul 06 14:13:01.247791 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.247748 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-9d6vz_dc19b82f-3557-445e-bcd7-33766be1a5cc/kube-rbac-proxy-main/0.log" Jul 06 14:13:01.282114 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.281943 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-9d6vz_dc19b82f-3557-445e-bcd7-33766be1a5cc/kube-rbac-proxy-self/0.log" Jul 06 14:13:01.321027 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.320978 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-cc4cccc59-rvbwb_bd29ce12-fb44-4780-ad0b-638ced353e4b/metrics-server/0.log" Jul 06 14:13:01.353920 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.351200 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-6hlg8_9df69d67-9d32-4bc1-b99e-3891f527d517/monitoring-plugin/0.log" Jul 06 14:13:01.397919 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.396927 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-82zbt_81ffeb38-a7fb-49a5-a723-21a8a5ed50fa/node-exporter/0.log" Jul 06 14:13:01.435221 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.435192 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-82zbt_81ffeb38-a7fb-49a5-a723-21a8a5ed50fa/kube-rbac-proxy/0.log" Jul 06 14:13:01.460776 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.460744 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-82zbt_81ffeb38-a7fb-49a5-a723-21a8a5ed50fa/init-textfile/0.log" Jul 06 14:13:01.711433 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.711031 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-68h6b_dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb/kube-rbac-proxy-main/0.log" Jul 06 14:13:01.745527 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.745496 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-68h6b_dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb/kube-rbac-proxy-self/0.log" Jul 06 14:13:01.775061 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.775024 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-68h6b_dfd54d49-dc01-4f0b-b7f3-56eb2356a5cb/openshift-state-metrics/0.log" Jul 06 14:13:01.819816 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.819733 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8a53b7e-1165-4dac-be6e-2923a7a57aa9/prometheus/0.log" Jul 06 14:13:01.846164 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.846116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8a53b7e-1165-4dac-be6e-2923a7a57aa9/config-reloader/0.log" Jul 06 14:13:01.875959 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.875921 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8a53b7e-1165-4dac-be6e-2923a7a57aa9/thanos-sidecar/0.log" Jul 06 14:13:01.912989 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.912963 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8a53b7e-1165-4dac-be6e-2923a7a57aa9/kube-rbac-proxy-web/0.log" Jul 06 14:13:01.942516 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.942490 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8a53b7e-1165-4dac-be6e-2923a7a57aa9/kube-rbac-proxy/0.log" Jul 06 14:13:01.969534 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.969501 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8a53b7e-1165-4dac-be6e-2923a7a57aa9/kube-rbac-proxy-thanos/0.log" Jul 06 14:13:01.996237 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:01.996207 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f8a53b7e-1165-4dac-be6e-2923a7a57aa9/init-config-reloader/0.log" Jul 06 14:13:02.031972 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.031943 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-b688k_cc698a30-3594-49d6-830a-97122a99f6aa/prometheus-operator/0.log" Jul 06 14:13:02.056257 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.056228 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-b688k_cc698a30-3594-49d6-830a-97122a99f6aa/kube-rbac-proxy/0.log" Jul 06 14:13:02.087936 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.087836 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-gdmkp_380d419c-c9e9-44fa-abb6-fab97c53fb75/prometheus-operator-admission-webhook/0.log" Jul 06 14:13:02.121355 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.121321 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c44dd9c4c-7s9rl_e3acfa3e-43d9-4447-908b-0bc7f57c2e75/telemeter-client/0.log" Jul 06 14:13:02.145925 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.145879 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c44dd9c4c-7s9rl_e3acfa3e-43d9-4447-908b-0bc7f57c2e75/reload/0.log" Jul 06 14:13:02.173724 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.173701 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5c44dd9c4c-7s9rl_e3acfa3e-43d9-4447-908b-0bc7f57c2e75/kube-rbac-proxy/0.log" Jul 06 14:13:02.223396 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.223358 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cddf75cd8-kjz2v_4aed4c15-624e-4cc8-a125-9c89b331c50f/thanos-query/0.log" Jul 06 14:13:02.247642 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.247596 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cddf75cd8-kjz2v_4aed4c15-624e-4cc8-a125-9c89b331c50f/kube-rbac-proxy-web/0.log" Jul 06 14:13:02.271641 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.271577 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cddf75cd8-kjz2v_4aed4c15-624e-4cc8-a125-9c89b331c50f/kube-rbac-proxy/0.log" Jul 06 14:13:02.297219 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.297187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cddf75cd8-kjz2v_4aed4c15-624e-4cc8-a125-9c89b331c50f/prom-label-proxy/0.log" Jul 06 14:13:02.321943 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.321908 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cddf75cd8-kjz2v_4aed4c15-624e-4cc8-a125-9c89b331c50f/kube-rbac-proxy-rules/0.log" Jul 06 14:13:02.347163 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:02.347065 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cddf75cd8-kjz2v_4aed4c15-624e-4cc8-a125-9c89b331c50f/kube-rbac-proxy-metrics/0.log" Jul 06 14:13:04.383213 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.383178 2562 scope.go:117] "RemoveContainer" containerID="db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8" Jul 06 14:13:04.383816 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:13:04.383436 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:13:04.552921 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.552869 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct"] Jul 06 14:13:04.560109 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.560082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.565054 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.565016 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct"] Jul 06 14:13:04.619479 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.619442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-proc\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.619704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.619494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-lib-modules\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.619704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.619590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-sys\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.619704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.619623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-426ds\" (UniqueName: \"kubernetes.io/projected/0350b448-2121-4a7c-be8d-f7329c82485a-kube-api-access-426ds\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.619704 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.619661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-podres\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720301 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-lib-modules\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720301 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720227 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-sys\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720301 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-426ds\" (UniqueName: \"kubernetes.io/projected/0350b448-2121-4a7c-be8d-f7329c82485a-kube-api-access-426ds\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720619 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720306 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-podres\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720619 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720343 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-sys\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720619 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720463 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-podres\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720619 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720469 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-lib-modules\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720619 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720504 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-proc\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.720619 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.720590 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0350b448-2121-4a7c-be8d-f7329c82485a-proc\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.730091 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.730053 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-426ds\" (UniqueName: \"kubernetes.io/projected/0350b448-2121-4a7c-be8d-f7329c82485a-kube-api-access-426ds\") pod \"perf-node-gather-daemonset-5fbct\" (UID: \"0350b448-2121-4a7c-be8d-f7329c82485a\") " pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:04.786240 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.786212 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-696bb467c6-59g9z_33fe9478-5f00-4490-ba78-e12e226c18ff/console/0.log" Jul 06 14:13:04.874696 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:04.874650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:05.043712 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:05.043678 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct"] Jul 06 14:13:05.047881 ip-10-0-129-92 kubenswrapper[2562]: W0706 14:13:05.047847 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0350b448_2121_4a7c_be8d_f7329c82485a.slice/crio-2ccde88ee19f448e0b85df4b6098ad35b2198d70568e64a5896b4061282c7ccb WatchSource:0}: Error finding container 2ccde88ee19f448e0b85df4b6098ad35b2198d70568e64a5896b4061282c7ccb: Status 404 returned error can't find the container with id 2ccde88ee19f448e0b85df4b6098ad35b2198d70568e64a5896b4061282c7ccb Jul 06 14:13:05.162257 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:05.162207 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" event={"ID":"0350b448-2121-4a7c-be8d-f7329c82485a","Type":"ContainerStarted","Data":"2ccde88ee19f448e0b85df4b6098ad35b2198d70568e64a5896b4061282c7ccb"} Jul 06 14:13:06.172157 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:06.172116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" event={"ID":"0350b448-2121-4a7c-be8d-f7329c82485a","Type":"ContainerStarted","Data":"bed269ab566ee6615756260c77cf0042a1f94819de2f2fdbd77964c7e42d152d"} Jul 06 14:13:06.172571 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:06.172180 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:06.192016 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:06.191963 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" podStartSLOduration=2.191947056 podStartE2EDuration="2.191947056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:13:06.190667231 +0000 UTC m=+737.379716279" watchObservedRunningTime="2026-07-06 14:13:06.191947056 +0000 UTC m=+737.380996103" Jul 06 14:13:06.503966 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:06.503883 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d7kjp_d2a56be5-b85c-4abb-828b-ea42f00a8f7d/dns/0.log" Jul 06 14:13:06.527387 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:06.527359 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d7kjp_d2a56be5-b85c-4abb-828b-ea42f00a8f7d/kube-rbac-proxy/0.log" Jul 06 14:13:06.671379 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:06.671345 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7fdpb_98ec5735-4c53-4696-b8e0-2dfc7ea09cbc/dns-node-resolver/0.log" Jul 06 14:13:07.342447 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:07.342407 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rvnvx_09688289-2a10-441b-bc83-c7453ea6bb61/node-ca/0.log" Jul 06 14:13:08.350541 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:08.350508 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gbst7_42f86e98-10ae-4a13-9639-c367f911d3d8/istio-proxy/0.log" Jul 06 14:13:08.481467 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:08.481435 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-nmqnb_8f349803-9b28-4683-b81f-9bb0f4b0639a/istio-proxy/0.log" Jul 06 14:13:09.169298 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:09.169260 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bdv7g_ddafc31d-a61f-4e4d-87c9-d70e43da0637/serve-healthcheck-canary/0.log" Jul 06 14:13:09.729346 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:09.729304 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9d579_691adf51-ea2f-4aca-aaea-c357787cac97/kube-rbac-proxy/0.log" Jul 06 14:13:09.756459 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:09.756435 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9d579_691adf51-ea2f-4aca-aaea-c357787cac97/exporter/0.log" Jul 06 14:13:09.785618 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:09.785591 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9d579_691adf51-ea2f-4aca-aaea-c357787cac97/extractor/0.log" Jul 06 14:13:12.086707 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.086676 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-sbz6m_e52cb6c2-5732-4193-9f0b-7277f55177fd/manager/0.log" Jul 06 14:13:12.126918 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.126870 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d8vl9_5693d981-477d-458d-a525-938e6f8724f9/manager/0.log" Jul 06 14:13:12.163402 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.163378 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-795f8494b4-dcsk9_67ad67f1-49f1-4303-a827-a0c7cd898805/manager/4.log" Jul 06 14:13:12.163545 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.163512 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-795f8494b4-dcsk9_67ad67f1-49f1-4303-a827-a0c7cd898805/manager/4.log" Jul 06 14:13:12.191545 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.191519 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f748f5f45-9r5lk_cc3d1864-d649-4722-a99a-da0adc569d33/manager/4.log" Jul 06 14:13:12.195146 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.195121 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f748f5f45-9r5lk_cc3d1864-d649-4722-a99a-da0adc569d33/manager/4.log" Jul 06 14:13:12.196415 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.196394 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lzps2/perf-node-gather-daemonset-5fbct" Jul 06 14:13:12.344104 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.344026 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d88dcf597-kjrrq_bc80f37c-c88a-4e80-9f07-a321b1e8bfb1/manager/0.log" Jul 06 14:13:12.378690 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.378662 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d88dcf597-kjrrq_bc80f37c-c88a-4e80-9f07-a321b1e8bfb1/copy-manifests/0.log" Jul 06 14:13:12.550113 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:12.550080 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-fqmhv_ac7461fc-4773-484a-867c-0306504a0a9e/postgres/0.log" Jul 06 14:13:13.383354 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:13.383273 2562 scope.go:117] "RemoveContainer" containerID="a80c1eda8159cb5816e040e5ba0c017f4966e6da13cc3da19bc4e7088518fa37" Jul 06 14:13:13.383854 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:13:13.383539 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-795f8494b4-dcsk9_opendatahub(67ad67f1-49f1-4303-a827-a0c7cd898805)\"" pod="opendatahub/maas-controller-795f8494b4-dcsk9" podUID="67ad67f1-49f1-4303-a827-a0c7cd898805" Jul 06 14:13:13.917976 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:13.917937 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7456975fc5-8qt4f_a134ed3b-b57c-4d88-a826-2a9172c673bf/manager/0.log" Jul 06 14:13:13.986684 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:13.986653 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2djb2_0364c89b-5576-4441-8e1f-79c3af2f0e5e/openshift-lws-operator/0.log" Jul 06 14:13:14.566690 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:14.566649 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gbst7_42f86e98-10ae-4a13-9639-c367f911d3d8/istio-proxy/0.log" Jul 06 14:13:14.707143 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:14.707111 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-nmqnb_8f349803-9b28-4683-b81f-9bb0f4b0639a/istio-proxy/0.log" Jul 06 14:13:15.783842 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:15.783805 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gbst7_42f86e98-10ae-4a13-9639-c367f911d3d8/istio-proxy/0.log" Jul 06 14:13:15.899391 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:15.899359 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-nmqnb_8f349803-9b28-4683-b81f-9bb0f4b0639a/istio-proxy/0.log" Jul 06 14:13:19.388310 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:19.388272 2562 scope.go:117] "RemoveContainer" containerID="db82217c0962b75e9cd12236871d9f89ac1879ac3234dc4cd1b4c6cb5d95bab8" Jul 06 14:13:19.388856 ip-10-0-129-92 kubenswrapper[2562]: E0706 14:13:19.388524 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=maas-controller-7f748f5f45-9r5lk_opendatahub(cc3d1864-d649-4722-a99a-da0adc569d33)\"" pod="opendatahub/maas-controller-7f748f5f45-9r5lk" podUID="cc3d1864-d649-4722-a99a-da0adc569d33" Jul 06 14:13:21.164978 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.164947 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fv7dr_955a1098-f214-44eb-bcb0-c73ccaba2be5/kube-multus-additional-cni-plugins/0.log" Jul 06 14:13:21.195558 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.195499 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fv7dr_955a1098-f214-44eb-bcb0-c73ccaba2be5/egress-router-binary-copy/0.log" Jul 06 14:13:21.228920 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.228860 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fv7dr_955a1098-f214-44eb-bcb0-c73ccaba2be5/cni-plugins/0.log" Jul 06 14:13:21.255788 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.255760 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fv7dr_955a1098-f214-44eb-bcb0-c73ccaba2be5/bond-cni-plugin/0.log" Jul 06 14:13:21.281577 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.281553 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fv7dr_955a1098-f214-44eb-bcb0-c73ccaba2be5/routeoverride-cni/0.log" Jul 06 14:13:21.307939 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.307890 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fv7dr_955a1098-f214-44eb-bcb0-c73ccaba2be5/whereabouts-cni-bincopy/0.log" Jul 06 14:13:21.335391 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.335361 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fv7dr_955a1098-f214-44eb-bcb0-c73ccaba2be5/whereabouts-cni/0.log" Jul 06 14:13:21.765886 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.765854 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-drpr8_f12f1a7e-9909-4508-b69c-095ffc809462/kube-multus/0.log" Jul 06 14:13:21.862164 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.862127 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ll8js_09458f3b-e5f7-4f3d-b621-5396d4176382/network-metrics-daemon/0.log" Jul 06 14:13:21.886351 ip-10-0-129-92 kubenswrapper[2562]: I0706 14:13:21.886308 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ll8js_09458f3b-e5f7-4f3d-b621-5396d4176382/kube-rbac-proxy/0.log"