Jul 16 17:12:23.523869 ip-10-0-133-160 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 17:12:23.523881 ip-10-0-133-160 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 17:12:23.523889 ip-10-0-133-160 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 17:12:23.524154 ip-10-0-133-160 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 17:12:33.543078 ip-10-0-133-160 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 17:12:33.543097 ip-10-0-133-160 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e26a9c626bc34f6b98c61be44b6aa3b7 -- Jul 16 17:14:37.035086 ip-10-0-133-160 systemd[1]: Starting Kubernetes Kubelet... Jul 16 17:14:37.420599 ip-10-0-133-160 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:14:37.420599 ip-10-0-133-160 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 17:14:37.420599 ip-10-0-133-160 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:14:37.420599 ip-10-0-133-160 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 17:14:37.420599 ip-10-0-133-160 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 17:14:37.422482 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.422336 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 17:14:37.427279 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427263 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:37.427279 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427279 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427283 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427286 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427290 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427292 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427295 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427298 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427302 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427304 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427307 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427309 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427312 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427314 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427317 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427320 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427322 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427325 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427327 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427330 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:37.427334 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427333 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427336 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427339 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427342 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427345 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427348 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427350 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427353 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427356 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427358 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427361 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427364 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427367 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427369 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427371 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427374 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427376 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427379 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427381 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:37.427774 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427386 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427389 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427392 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427394 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427397 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427399 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427402 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427405 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427407 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427410 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427413 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427415 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427418 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427420 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427423 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427426 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427429 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427432 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427434 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427437 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:37.428259 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427439 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427442 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427446 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427449 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427453 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427456 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427459 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427462 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427465 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427468 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427470 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427473 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427476 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427478 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427481 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427484 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427486 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427490 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427493 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:37.428749 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427496 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427498 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427501 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427503 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427506 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427508 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427511 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427514 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427517 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427520 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427523 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427526 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427528 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427531 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427534 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427536 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427539 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427542 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427544 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427547 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:37.429205 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.427549 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428261 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428268 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428271 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428274 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428277 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428280 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428283 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428286 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428288 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428291 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428293 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428296 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428298 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428301 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428304 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428306 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428309 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428311 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428314 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:37.429674 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428318 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428321 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428324 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428326 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428329 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428332 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428335 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428338 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428340 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428343 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428345 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428348 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428350 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428353 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428355 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428358 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428360 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428365 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428368 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:37.430201 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428372 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428375 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428378 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428381 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428383 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428386 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428388 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428390 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428393 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428396 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428399 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428401 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428405 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428408 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428411 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428414 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428416 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428419 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428422 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:37.430690 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428424 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428427 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428430 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428432 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428435 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428438 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428441 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428443 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428445 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428448 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428450 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428453 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428455 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428459 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428463 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428466 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428468 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428471 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428473 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428476 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:37.431161 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428478 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428482 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428484 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428487 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428490 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428492 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428495 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428498 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428500 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428503 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428505 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428508 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428510 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428513 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428516 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428518 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428521 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428523 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428527 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428530 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:37.431635 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428533 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428536 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428939 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428944 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428947 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428951 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428955 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428958 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428960 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428963 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428966 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428969 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428972 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428975 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428977 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428980 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428983 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428986 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428989 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428991 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:37.432143 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428994 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428996 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.428999 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429002 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429004 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429007 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429009 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429012 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429015 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429017 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429020 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429022 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429024 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429027 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429029 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429032 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429035 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429037 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429040 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429043 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:37.432620 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429045 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429048 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429051 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429053 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429056 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429058 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429061 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429063 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429066 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429069 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429071 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429074 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429077 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429080 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429082 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429085 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429088 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429090 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429093 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429095 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:37.433121 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429098 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429101 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429103 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429105 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429108 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429110 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429112 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429115 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429118 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429121 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429123 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429127 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429131 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429136 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429139 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429142 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429145 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429148 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429151 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:37.433601 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429153 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429157 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429159 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429162 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429167 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429170 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429173 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429176 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429178 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429180 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429183 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429186 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429188 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429191 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429193 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429197 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429200 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429203 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429206 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:37.434055 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429209 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429212 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429214 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429296 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429306 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429315 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429321 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429330 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429334 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429338 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429342 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429345 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429348 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429352 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429357 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429360 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429363 2577 flags.go:64] FLAG: --cgroup-root="" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429366 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429369 2577 flags.go:64] FLAG: --client-ca-file="" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429372 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429375 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429381 2577 flags.go:64] FLAG: --cluster-domain="" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429384 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429387 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 17:14:37.434496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429391 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429394 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429398 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429401 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429404 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429407 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429410 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429413 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429416 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429419 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429423 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429427 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429430 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429433 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429435 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429440 2577 flags.go:64] FLAG: --enable-server="true" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429444 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429448 2577 flags.go:64] FLAG: --event-burst="100" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429451 2577 flags.go:64] FLAG: --event-qps="50" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429454 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429457 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429460 2577 flags.go:64] FLAG: --eviction-hard="" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429465 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429468 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429471 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 17:14:37.435092 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429474 2577 flags.go:64] FLAG: --eviction-soft="" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429477 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429480 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429483 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429486 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429488 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429491 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429494 2577 flags.go:64] FLAG: --feature-gates="" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429498 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429501 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429504 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429507 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429510 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429514 2577 flags.go:64] FLAG: --help="false" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429516 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429519 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429522 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429525 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429528 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429532 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429535 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429537 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429541 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 17:14:37.435678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429544 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429548 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429551 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429554 2577 flags.go:64] FLAG: --kube-reserved="" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429557 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429559 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429564 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429566 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429569 2577 flags.go:64] FLAG: --lock-file="" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429572 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429575 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429578 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429583 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429586 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429589 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429592 2577 flags.go:64] FLAG: --logging-format="text" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429594 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429597 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429600 2577 flags.go:64] FLAG: --manifest-url="" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429603 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429608 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429611 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429615 2577 flags.go:64] FLAG: --max-pods="110" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429617 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429621 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 17:14:37.436333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429624 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429627 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429630 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429633 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429636 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429644 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429648 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429651 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429654 2577 flags.go:64] FLAG: --pod-cidr="" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429658 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429664 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429666 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429670 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429674 2577 flags.go:64] FLAG: --port="10250" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429677 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429680 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f67b5363e3ffa1b" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429683 2577 flags.go:64] FLAG: --qos-reserved="" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429686 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429689 2577 flags.go:64] FLAG: --register-node="true" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429692 2577 flags.go:64] FLAG: --register-with-taints="" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429696 2577 flags.go:64] FLAG: --registry-burst="10" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429699 2577 flags.go:64] FLAG: --registry-qps="5" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429702 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429704 2577 flags.go:64] FLAG: --reserved-memory="" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429708 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 17:14:37.436965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429711 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429714 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429717 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429720 2577 flags.go:64] FLAG: --runonce="false" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429723 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429727 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429729 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429732 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429735 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429738 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429741 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429744 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429747 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429749 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429753 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429755 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429758 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429762 2577 flags.go:64] FLAG: --system-cgroups="" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429764 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429770 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429774 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429777 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429781 2577 flags.go:64] FLAG: --tls-min-version="" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429784 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429798 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 17:14:37.437550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429801 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429804 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429807 2577 flags.go:64] FLAG: --v="2" Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429811 2577 flags.go:64] FLAG: --version="false" Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429815 2577 flags.go:64] FLAG: --vmodule="" Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429819 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.429823 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429939 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429943 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429945 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429948 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429951 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429953 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429956 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429958 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429961 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429964 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429967 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429969 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429971 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429974 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:37.438206 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429977 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429979 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429982 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429985 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429988 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429990 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429994 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429997 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.429999 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430002 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430004 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430007 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430009 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430012 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430014 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430017 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430020 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430022 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430025 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430029 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:37.438709 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430033 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430036 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430039 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430042 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430044 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430047 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430050 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430053 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430056 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430058 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430061 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430063 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430066 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430069 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430071 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430074 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430076 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430079 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430084 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430087 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:37.439269 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430089 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430092 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430094 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430097 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430099 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430102 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430104 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430107 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430109 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430112 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430114 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430117 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430119 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430123 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430126 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430130 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430134 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430137 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430140 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:37.440150 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430143 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430145 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430148 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430150 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430152 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430155 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430158 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430160 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430163 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430165 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430168 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430172 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430175 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430177 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430180 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430182 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430185 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430187 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430190 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430192 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:37.440658 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430195 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430197 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430200 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430203 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430206 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.430208 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.430704 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.437733 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.437748 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.437765 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.437771 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437853 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437859 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437862 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437865 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:37.441208 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437868 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437871 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437873 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437876 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437879 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437882 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437884 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437886 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437897 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437900 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437903 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437906 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437908 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437911 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437914 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437916 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437922 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437925 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437928 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437931 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:37.441613 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437933 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437937 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437941 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437944 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437948 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437952 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437961 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437964 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437967 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437969 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437972 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437975 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437977 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437980 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437982 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437985 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437987 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437990 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437992 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:37.442149 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437995 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.437997 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438008 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438011 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438013 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438016 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438019 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438021 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438024 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438026 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438029 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438031 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438033 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438036 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438039 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438041 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438043 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438046 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438049 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:37.442643 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438052 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438055 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438057 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438060 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438063 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438065 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438068 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438071 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438073 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438076 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438080 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438083 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438086 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438089 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438091 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438094 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438102 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438105 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438108 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438110 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:37.443169 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438113 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438115 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438118 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438120 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438123 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438125 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438127 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438130 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438132 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438135 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438137 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438140 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438143 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438146 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438148 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438151 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:37.443765 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438154 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.438160 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438319 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438323 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438326 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438329 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438331 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438334 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438337 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438339 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438342 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438344 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438347 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438357 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438360 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 17:14:37.444283 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438363 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438367 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438370 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438372 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438375 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438378 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438381 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438383 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438386 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438389 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438391 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438394 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438397 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438400 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438402 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438405 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438409 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438412 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438415 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 17:14:37.444678 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438419 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438422 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438425 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438428 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438430 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438433 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438435 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438438 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438440 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438442 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438445 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438447 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438456 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438458 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438461 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438463 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438466 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438468 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438471 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438473 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 17:14:37.445195 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438476 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438479 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438481 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438484 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438486 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438489 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438491 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438493 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438496 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438498 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438501 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438503 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438506 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438508 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438511 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438513 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438515 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438518 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438520 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438523 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 17:14:37.445735 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438525 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438528 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438530 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438533 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438535 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438543 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438546 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438548 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438551 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438553 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438556 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438558 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438561 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438563 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438566 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438568 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438571 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438573 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438576 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438578 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 17:14:37.446572 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438580 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438583 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438585 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438588 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438590 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438592 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:37.438595 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.438600 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.438719 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.441508 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.442883 2577 server.go:1019] "Starting client certificate rotation" Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.442998 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 17:14:37.447298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.443030 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 17:14:37.448206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.448178 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 17:14:37.449662 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.449640 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 17:14:37.460272 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.460249 2577 log.go:25] "Validated CRI v1 runtime API" Jul 16 17:14:37.462236 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.462222 2577 log.go:25] "Validated CRI v1 image API" Jul 16 17:14:37.464708 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.464685 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 17:14:37.467966 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.467945 2577 fs.go:135] Filesystem UUIDs: map[2bf8fe4f-5319-4944-bfc0-bb8cfc01c339:/dev/nvme0n1p4 6ce5a317-fe86-40a2-a604-dd6a870a502a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 16 17:14:37.468041 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.467964 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 17:14:37.474271 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.474150 2577 manager.go:216] Machine: {Timestamp:2026-07-16 17:14:37.472753486 +0000 UTC m=+0.322259863 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098726 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a8080e9769ec0ee43b6925dd8888e SystemUUID:ec2a8080-e976-9ec0-ee43-b6925dd8888e BootID:e26a9c62-6bc3-4f6b-98c6-1be44b6aa3b7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:a5:42:65:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:a5:42:65:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:15:66:61:fe:82 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 17:14:37.474271 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.474264 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 17:14:37.474383 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.474350 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 17:14:37.477750 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.477723 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 17:14:37.477916 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.477754 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 17:14:37.478622 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.478612 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 17:14:37.478659 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.478625 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 17:14:37.478659 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.478633 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 17:14:37.479434 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.479422 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 17:14:37.480608 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.480595 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 17:14:37.482594 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.482576 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 17:14:37.483220 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483196 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 16 17:14:37.483408 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483397 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 17:14:37.483450 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483434 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 16 17:14:37.483480 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483449 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 17:14:37.483480 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483474 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 17:14:37.483544 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483489 2577 kubelet.go:398] "Adding apiserver pod source" Jul 16 17:14:37.483544 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483502 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 17:14:37.483970 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483951 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 17:14:37.484019 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.483976 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 17:14:37.486189 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.486160 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 17:14:37.489362 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.489338 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 17:14:37.489458 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.489391 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 17:14:37.490835 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490817 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490840 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490847 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490853 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490862 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490871 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490886 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490894 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490901 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 17:14:37.490912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490907 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 17:14:37.491214 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490922 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 17:14:37.491214 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490928 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 17:14:37.491214 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490955 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 17:14:37.491214 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.490961 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 17:14:37.494237 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.494213 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 17:14:37.494326 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.494269 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 17:14:37.494326 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.494299 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 17:14:37.494571 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.494557 2577 server.go:1268] "Started kubelet" Jul 16 17:14:37.494657 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.494627 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 17:14:37.494718 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.494645 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 17:14:37.494761 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.494746 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 17:14:37.495533 ip-10-0-133-160 systemd[1]: Started Kubernetes Kubelet. Jul 16 17:14:37.497316 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.497297 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 16 17:14:37.499191 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.499175 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 17:14:37.499715 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.499685 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 17:14:37.501331 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.500349 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-160.ec2.internal.18c2d486b9787489 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-160.ec2.internal,UID:ip-10-0-133-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-160.ec2.internal,},FirstTimestamp:2026-07-16 17:14:37.494523017 +0000 UTC m=+0.344029376,LastTimestamp:2026-07-16 17:14:37.494523017 +0000 UTC m=+0.344029376,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-160.ec2.internal,}" Jul 16 17:14:37.503765 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.503744 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 17:14:37.503765 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.503750 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 17:14:37.503968 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.503951 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:37.504455 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.504384 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 17:14:37.504455 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.504404 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 17:14:37.504577 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.504495 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 17:14:37.504628 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.504586 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 16 17:14:37.504628 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.504593 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 16 17:14:37.506109 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.506094 2577 factory.go:55] Registering systemd factory Jul 16 17:14:37.506202 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.506113 2577 factory.go:223] Registration of the systemd container factory successfully Jul 16 17:14:37.506367 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.506351 2577 factory.go:153] Registering CRI-O factory Jul 16 17:14:37.506437 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.506370 2577 factory.go:223] Registration of the crio container factory successfully Jul 16 17:14:37.506437 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.506412 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 17:14:37.506437 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.506430 2577 factory.go:103] Registering Raw factory Jul 16 17:14:37.506593 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.506441 2577 manager.go:1196] Started watching for new ooms in manager Jul 16 17:14:37.507111 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.507096 2577 manager.go:349] Starting recovery of all containers Jul 16 17:14:37.509943 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.509910 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 17:14:37.510051 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.509936 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 17:14:37.520689 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.520662 2577 manager.go:354] Recovery completed Jul 16 17:14:37.521762 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.521737 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p42dc" Jul 16 17:14:37.526716 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.526700 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:37.528678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.528663 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:37.528740 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.528694 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:37.528740 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.528704 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:37.529392 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.529371 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 17:14:37.529392 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.529384 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 17:14:37.529520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.529400 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 16 17:14:37.529520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.529401 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:37.530459 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.530433 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p42dc" Jul 16 17:14:37.530551 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.530496 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:37.530551 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.530524 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:37.530551 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.530537 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:37.530735 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.530721 2577 policy_none.go:49] "None policy: Start" Jul 16 17:14:37.530769 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.530745 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 17:14:37.530769 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.530763 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 17:14:37.531834 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.531741 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-160.ec2.internal.18c2d486bb81a24c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-160.ec2.internal,UID:ip-10-0-133-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-160.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-160.ec2.internal,},FirstTimestamp:2026-07-16 17:14:37.528678988 +0000 UTC m=+0.378185346,LastTimestamp:2026-07-16 17:14:37.528678988 +0000 UTC m=+0.378185346,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-160.ec2.internal,}" Jul 16 17:14:37.532025 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.532007 2577 policy_none.go:47] "Start" Jul 16 17:14:37.568603 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.568449 2577 manager.go:337] "Starting Device Plugin manager" Jul 16 17:14:37.568730 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.568650 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 17:14:37.568730 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.568666 2577 server.go:87] "Starting device plugin registration server" Jul 16 17:14:37.568969 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.568956 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 17:14:37.569027 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.568970 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 17:14:37.569117 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.569096 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 17:14:37.569220 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.569191 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 17:14:37.569220 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.569202 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 17:14:37.569846 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.569804 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 17:14:37.569919 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.569887 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:37.670388 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.670335 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:37.671601 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.671495 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:37.671601 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.671525 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:37.671601 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.671541 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:37.671601 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.671575 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.679351 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.679325 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 17:14:37.680747 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.680729 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 17:14:37.680836 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.680776 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 17:14:37.680836 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.680809 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 17:14:37.680939 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.680857 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 17:14:37.684458 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.684436 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:37.693569 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.693551 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.693653 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.693575 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-160.ec2.internal\": node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:37.783561 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.783513 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal"] Jul 16 17:14:37.783749 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.783608 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:37.784695 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.784678 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:37.784805 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.784710 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:37.784805 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.784721 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:37.786079 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786065 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:37.786221 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786207 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.786259 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786239 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:37.786896 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786871 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:37.786896 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786892 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:37.787018 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786900 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:37.787018 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786917 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:37.787018 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786933 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:37.787018 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.786935 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:37.788144 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.788130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.788196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.788157 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 17:14:37.788890 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.788849 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientMemory" Jul 16 17:14:37.788890 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.788879 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 17:14:37.788890 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.788892 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeHasSufficientPID" Jul 16 17:14:37.806005 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.805979 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d01a527a86fa60749d30ac5f9ca8fcf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal\" (UID: \"d01a527a86fa60749d30ac5f9ca8fcf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.806143 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.806013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d01a527a86fa60749d30ac5f9ca8fcf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal\" (UID: \"d01a527a86fa60749d30ac5f9ca8fcf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.806143 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.806040 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df2695e99c9c68ac5fa2b0530a286936-config\") pod \"kube-apiserver-proxy-ip-10-0-133-160.ec2.internal\" (UID: \"df2695e99c9c68ac5fa2b0530a286936\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.807498 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.807479 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-160.ec2.internal\" not found" node="ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.811100 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.811082 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-160.ec2.internal\" not found" node="ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.812897 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.812860 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:37.906916 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.906881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d01a527a86fa60749d30ac5f9ca8fcf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal\" (UID: \"d01a527a86fa60749d30ac5f9ca8fcf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.906916 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.906920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d01a527a86fa60749d30ac5f9ca8fcf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal\" (UID: \"d01a527a86fa60749d30ac5f9ca8fcf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.907150 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.906947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df2695e99c9c68ac5fa2b0530a286936-config\") pod \"kube-apiserver-proxy-ip-10-0-133-160.ec2.internal\" (UID: \"df2695e99c9c68ac5fa2b0530a286936\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.907150 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.906992 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d01a527a86fa60749d30ac5f9ca8fcf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal\" (UID: \"d01a527a86fa60749d30ac5f9ca8fcf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.907150 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.907005 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d01a527a86fa60749d30ac5f9ca8fcf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal\" (UID: \"d01a527a86fa60749d30ac5f9ca8fcf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.907150 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:37.906998 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df2695e99c9c68ac5fa2b0530a286936-config\") pod \"kube-apiserver-proxy-ip-10-0-133-160.ec2.internal\" (UID: \"df2695e99c9c68ac5fa2b0530a286936\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" Jul 16 17:14:37.914041 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:37.914020 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:38.014551 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.014457 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:38.111105 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.111076 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:38.113313 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.113295 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" Jul 16 17:14:38.115097 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.114845 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:38.215548 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.215517 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:38.316332 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.316255 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-160.ec2.internal\" not found" Jul 16 17:14:38.398935 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.398913 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:38.405001 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.404972 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" Jul 16 17:14:38.426168 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.426140 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 17:14:38.428356 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.428337 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" Jul 16 17:14:38.442419 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.442384 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 17:14:38.442547 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.442518 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 17:14:38.442594 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.442548 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 17:14:38.442594 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.442549 2577 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://ad683cc5765bd41568caf2dbd38220a0-0b39a665eecef20c.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.133.160:47842->3.230.207.51:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" Jul 16 17:14:38.484223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.484195 2577 apiserver.go:52] "Watching apiserver" Jul 16 17:14:38.495596 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.495568 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:38.498990 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.498966 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 17:14:38.500273 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.500248 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb","openshift-cluster-node-tuning-operator/tuned-spdgx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal","openshift-multus/multus-additional-cni-plugins-t9r69","openshift-network-diagnostics/network-check-target-6cn29","openshift-dns/node-resolver-7w5mz","openshift-image-registry/node-ca-rv7rf","openshift-multus/multus-zl4qh","openshift-multus/network-metrics-daemon-27zcc","openshift-network-operator/iptables-alerter-7bfv6","openshift-ovn-kubernetes/ovnkube-node-7wsjz","kube-system/global-pull-secret-syncer-rp2zv","kube-system/konnectivity-agent-fjqs9","kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal"] Jul 16 17:14:38.501630 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.501609 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:38.501733 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.501673 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:38.502706 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.502692 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.503800 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.503780 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.503909 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.503894 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 17:14:38.505047 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.505028 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.505706 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.505687 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 17:14:38.505706 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.505703 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 17:14:38.505847 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.505685 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 17:14:38.505847 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.505790 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 17:14:38.506159 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.506137 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.506942 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.506928 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 17:14:38.507009 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.506934 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 17:14:38.507009 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.506965 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lfsv5\"" Jul 16 17:14:38.507365 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.507349 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.508542 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.508527 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.509222 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509199 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 17:14:38.509322 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509226 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qzcvc\"" Jul 16 17:14:38.509322 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509277 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 17:14:38.509322 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509289 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 17:14:38.509465 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509308 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 17:14:38.509526 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509508 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 17:14:38.509579 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509567 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 17:14:38.509729 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509712 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:38.509851 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509756 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 17:14:38.509851 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.509790 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:38.509851 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509840 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 17:14:38.510031 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.509889 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hmzd2\"" Jul 16 17:14:38.510773 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.510756 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 17:14:38.510909 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.510891 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 17:14:38.511047 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.511030 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.511113 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.511077 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bg6pc\"" Jul 16 17:14:38.511113 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.511084 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 17:14:38.511274 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.511082 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 17:14:38.511631 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.511617 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 17:14:38.511698 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.511631 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gxcn8\"" Jul 16 17:14:38.512478 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.512457 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.513233 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513218 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pb258\"" Jul 16 17:14:38.513233 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-systemd\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513344 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513248 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-etc-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513344 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-env-overrides\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513344 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6xjj\" (UniqueName: \"kubernetes.io/projected/0bd348bf-425d-45d0-8fe0-a91dfc261aae-kube-api-access-p6xjj\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.513344 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513334 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-run-netns\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513483 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-cni-bin\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513483 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovnkube-config\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513483 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovn-node-metrics-cert\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513622 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.513622 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm9ks\" (UniqueName: \"kubernetes.io/projected/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-kube-api-access-qm9ks\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.513622 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513601 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-systemd-units\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513766 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513766 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.513766 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cnibin\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.513766 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513698 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.513766 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513700 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-os-release\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.513766 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.513758 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-var-lib-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-ovn\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-slash\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513868 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-log-socket\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513882 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-cni-netd\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513902 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bscxn\" (UniqueName: \"kubernetes.io/projected/3c274373-f9bb-42eb-bc62-4b247a4df6f0-kube-api-access-bscxn\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cni-binary-copy\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-node-log\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovnkube-script-lib\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.513997 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bd348bf-425d-45d0-8fe0-a91dfc261aae-serviceca\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.514460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.514027 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-system-cni-dir\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.514460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.514062 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:38.514460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.514105 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-kubelet\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.514460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.514121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bd348bf-425d-45d0-8fe0-a91dfc261aae-host\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.514460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.514151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.514460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.514198 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.515299 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.515283 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.517647 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.517629 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 17:14:38.519182 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519158 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:14:38.519282 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519190 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r9ts7\"" Jul 16 17:14:38.519282 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519219 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 17:14:38.519282 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519196 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 17:14:38.519429 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519409 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 17:14:38.519674 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519657 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 17:14:38.519799 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519780 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:14:38.519887 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519780 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 17:14:38.519887 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519839 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7gv4n\"" Jul 16 17:14:38.519887 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.519867 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dqx82\"" Jul 16 17:14:38.533031 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.532983 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 17:09:37 +0000 UTC" deadline="2028-01-20 20:02:33.90095956 +0000 UTC" Jul 16 17:14:38.533031 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.533027 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13274h47m55.36793597s" Jul 16 17:14:38.539716 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.539657 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 17:14:38.566184 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.566159 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5hgdh" Jul 16 17:14:38.577614 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.577592 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5hgdh" Jul 16 17:14:38.605386 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.605367 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 17:14:38.614571 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:38.614685 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614580 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-kubelet\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.614685 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614625 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-kubelet\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.614685 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.614685 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614661 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-sys-fs\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.614685 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614686 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-k8s-cni-cncf-io\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614705 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-hostroot\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-host\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-tuned\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614771 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-socket-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614826 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-system-cni-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-cnibin\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614859 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-etc-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614971 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-env-overrides\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.614993 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.614986 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-etc-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615002 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/056bd779-49dd-431f-89f8-0444aed7a025-iptables-alerter-script\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-cni-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615098 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/009ea160-3568-48ea-bd35-ec7b42f5d1a9-cni-binary-copy\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysctl-conf\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-run-netns\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-cni-bin\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615217 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/056bd779-49dd-431f-89f8-0444aed7a025-host-slash\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k424t\" (UniqueName: \"kubernetes.io/projected/056bd779-49dd-431f-89f8-0444aed7a025-kube-api-access-k424t\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615298 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kjpz\" (UniqueName: \"kubernetes.io/projected/5a8f2521-2a93-4462-89b6-e2d8aced60d5-kube-api-access-8kjpz\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615271 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-cni-bin\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615323 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-conf-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615350 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-modprobe-d\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-systemd-units\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615399 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.615484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cnibin\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615486 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-systemd-units\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615498 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-os-release\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615497 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-run-netns\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615527 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fe344e5a-2ae4-454a-8be3-bf168ef3ed1d-agent-certs\") pod \"konnectivity-agent-fjqs9\" (UID: \"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d\") " pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615533 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-ovn\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615564 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-env-overrides\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615587 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615592 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-kubelet\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615621 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-systemd\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615627 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cnibin\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615645 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-run\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615669 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fe344e5a-2ae4-454a-8be3-bf168ef3ed1d-konnectivity-ca\") pod \"konnectivity-agent-fjqs9\" (UID: \"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d\") " pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615681 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-ovn\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615688 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-os-release\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.616318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bscxn\" (UniqueName: \"kubernetes.io/projected/3c274373-f9bb-42eb-bc62-4b247a4df6f0-kube-api-access-bscxn\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615724 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615753 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e9f16af0-c690-4e38-955a-2aae0c93c87c-kubelet-config\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615777 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e9f16af0-c690-4e38-955a-2aae0c93c87c-dbus\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-etc-kubernetes\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615826 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-lib-modules\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615935 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n929q\" (UniqueName: \"kubernetes.io/projected/4c37b6ae-09dd-4587-a003-6ed07c747d40-kube-api-access-n929q\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-node-log\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.615984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovnkube-script-lib\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616053 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-node-log\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bd348bf-425d-45d0-8fe0-a91dfc261aae-serviceca\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-system-cni-dir\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616136 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8f2521-2a93-4462-89b6-e2d8aced60d5-tmp-dir\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-os-release\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616187 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bd348bf-425d-45d0-8fe0-a91dfc261aae-host\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616229 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stms6\" (UniqueName: \"kubernetes.io/projected/2a21a5f2-be62-4282-96ee-616aaf35f34d-kube-api-access-stms6\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.617223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616251 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bd348bf-425d-45d0-8fe0-a91dfc261aae-host\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-system-cni-dir\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616365 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a8f2521-2a93-4462-89b6-e2d8aced60d5-hosts-file\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-cni-multus\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-systemd\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616463 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0bd348bf-425d-45d0-8fe0-a91dfc261aae-serviceca\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616502 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-run-systemd\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6xjj\" (UniqueName: \"kubernetes.io/projected/0bd348bf-425d-45d0-8fe0-a91dfc261aae-kube-api-access-p6xjj\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616539 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8dw7\" (UniqueName: \"kubernetes.io/projected/26b0b5ae-c523-4603-b958-bf5e8b6017cc-kube-api-access-d8dw7\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-registration-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616594 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-sys\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616620 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c37b6ae-09dd-4587-a003-6ed07c747d40-tmp\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616625 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovnkube-script-lib\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovnkube-config\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovn-node-metrics-cert\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.617962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616704 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm9ks\" (UniqueName: \"kubernetes.io/projected/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-kube-api-access-qm9ks\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616729 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-device-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-daemon-config\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbdcm\" (UniqueName: \"kubernetes.io/projected/009ea160-3568-48ea-bd35-ec7b42f5d1a9-kube-api-access-rbdcm\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysctl-d\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-var-lib-kubelet\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-socket-dir-parent\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616935 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-cni-bin\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616962 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-kubernetes\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.616995 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-var-lib-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-netns\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-slash\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617128 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-var-lib-openvswitch\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-log-socket\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617172 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-log-socket\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617194 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovnkube-config\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617212 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-slash\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.618619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617252 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617257 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-cni-netd\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617324 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cni-binary-copy\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617343 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3c274373-f9bb-42eb-bc62-4b247a4df6f0-host-cni-netd\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysconfig\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617527 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-etc-selinux\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617555 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-multus-certs\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617948 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.619292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.617960 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-cni-binary-copy\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.620992 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.620970 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c274373-f9bb-42eb-bc62-4b247a4df6f0-ovn-node-metrics-cert\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.630232 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.630211 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:38.631011 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.630981 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm9ks\" (UniqueName: \"kubernetes.io/projected/b7f4f3eb-183d-4fd1-99e6-af152bd9aca0-kube-api-access-qm9ks\") pod \"multus-additional-cni-plugins-t9r69\" (UID: \"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0\") " pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.632572 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.632548 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:38.632572 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.632574 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:38.632719 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.632590 2577 projected.go:196] Error preparing data for projected volume kube-api-access-zqfx7 for pod openshift-network-diagnostics/network-check-target-6cn29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:38.632719 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.632712 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7 podName:97fdadb3-57ac-4ab3-8f58-dfacdff7eca1 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:39.132689244 +0000 UTC m=+1.982195590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zqfx7" (UniqueName: "kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7") pod "network-check-target-6cn29" (UID: "97fdadb3-57ac-4ab3-8f58-dfacdff7eca1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:38.634896 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.634849 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6xjj\" (UniqueName: \"kubernetes.io/projected/0bd348bf-425d-45d0-8fe0-a91dfc261aae-kube-api-access-p6xjj\") pod \"node-ca-rv7rf\" (UID: \"0bd348bf-425d-45d0-8fe0-a91dfc261aae\") " pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.635147 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.635127 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bscxn\" (UniqueName: \"kubernetes.io/projected/3c274373-f9bb-42eb-bc62-4b247a4df6f0-kube-api-access-bscxn\") pod \"ovnkube-node-7wsjz\" (UID: \"3c274373-f9bb-42eb-bc62-4b247a4df6f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.647303 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.647277 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2695e99c9c68ac5fa2b0530a286936.slice/crio-870cf3ec831f444602027647b808e8a53262739202a75656b52cd8eebe186f73 WatchSource:0}: Error finding container 870cf3ec831f444602027647b808e8a53262739202a75656b52cd8eebe186f73: Status 404 returned error can't find the container with id 870cf3ec831f444602027647b808e8a53262739202a75656b52cd8eebe186f73 Jul 16 17:14:38.648444 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.648421 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01a527a86fa60749d30ac5f9ca8fcf4.slice/crio-c5f4d97aa0911e2f502fd3c42fd26974d2b865b061e7a8ca2ff6e8b676e01077 WatchSource:0}: Error finding container c5f4d97aa0911e2f502fd3c42fd26974d2b865b061e7a8ca2ff6e8b676e01077: Status 404 returned error can't find the container with id c5f4d97aa0911e2f502fd3c42fd26974d2b865b061e7a8ca2ff6e8b676e01077 Jul 16 17:14:38.653605 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.653588 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:14:38.683780 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.683728 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" event={"ID":"d01a527a86fa60749d30ac5f9ca8fcf4","Type":"ContainerStarted","Data":"c5f4d97aa0911e2f502fd3c42fd26974d2b865b061e7a8ca2ff6e8b676e01077"} Jul 16 17:14:38.684879 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.684852 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" event={"ID":"df2695e99c9c68ac5fa2b0530a286936","Type":"ContainerStarted","Data":"870cf3ec831f444602027647b808e8a53262739202a75656b52cd8eebe186f73"} Jul 16 17:14:38.717968 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.717940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stms6\" (UniqueName: \"kubernetes.io/projected/2a21a5f2-be62-4282-96ee-616aaf35f34d-kube-api-access-stms6\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.718054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.717984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a8f2521-2a93-4462-89b6-e2d8aced60d5-hosts-file\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.718054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-cni-multus\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8dw7\" (UniqueName: \"kubernetes.io/projected/26b0b5ae-c523-4603-b958-bf5e8b6017cc-kube-api-access-d8dw7\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:38.718171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-registration-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.718171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-sys\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718076 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5a8f2521-2a93-4462-89b6-e2d8aced60d5-hosts-file\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.718171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c37b6ae-09dd-4587-a003-6ed07c747d40-tmp\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-cni-multus\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-registration-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.718171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718170 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-device-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-daemon-config\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718208 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-sys\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbdcm\" (UniqueName: \"kubernetes.io/projected/009ea160-3568-48ea-bd35-ec7b42f5d1a9-kube-api-access-rbdcm\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysctl-d\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-device-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-var-lib-kubelet\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718298 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-socket-dir-parent\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718328 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-cni-bin\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-kubernetes\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718379 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-netns\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718414 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysctl-d\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718349 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-var-lib-kubelet\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysconfig\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718484 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-socket-dir-parent\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718475 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-cni-bin\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.718512 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718487 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-etc-selinux\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-multus-certs\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718560 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-etc-selinux\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.718562 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718586 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-netns\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.718640 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret podName:e9f16af0-c690-4e38-955a-2aae0c93c87c nodeName:}" failed. No retries permitted until 2026-07-16 17:14:39.218621252 +0000 UTC m=+2.068127593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret") pod "global-pull-secret-syncer-rp2zv" (UID: "e9f16af0-c690-4e38-955a-2aae0c93c87c") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718582 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-kubernetes\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-multus-certs\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718658 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysconfig\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-daemon-config\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-sys-fs\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718905 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-k8s-cni-cncf-io\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718922 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-sys-fs\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.718965 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-hostroot\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719037 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-run-k8s-cni-cncf-io\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719068 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-hostroot\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-host\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.719403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-tuned\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-socket-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719125 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-host\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-system-cni-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-cnibin\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719260 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-socket-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-system-cni-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/056bd779-49dd-431f-89f8-0444aed7a025-iptables-alerter-script\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719312 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-cnibin\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719335 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-cni-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/009ea160-3568-48ea-bd35-ec7b42f5d1a9-cni-binary-copy\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysctl-conf\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/056bd779-49dd-431f-89f8-0444aed7a025-host-slash\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k424t\" (UniqueName: \"kubernetes.io/projected/056bd779-49dd-431f-89f8-0444aed7a025-kube-api-access-k424t\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719492 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kjpz\" (UniqueName: \"kubernetes.io/projected/5a8f2521-2a93-4462-89b6-e2d8aced60d5-kube-api-access-8kjpz\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-conf-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719537 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-sysctl-conf\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.720364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-modprobe-d\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719549 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-cni-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fe344e5a-2ae4-454a-8be3-bf168ef3ed1d-agent-certs\") pod \"konnectivity-agent-fjqs9\" (UID: \"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d\") " pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-kubelet\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719625 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-modprobe-d\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-systemd\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-run\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719688 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fe344e5a-2ae4-454a-8be3-bf168ef3ed1d-konnectivity-ca\") pod \"konnectivity-agent-fjqs9\" (UID: \"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d\") " pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719716 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e9f16af0-c690-4e38-955a-2aae0c93c87c-kubelet-config\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719785 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-multus-conf-dir\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719798 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e9f16af0-c690-4e38-955a-2aae0c93c87c-kubelet-config\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e9f16af0-c690-4e38-955a-2aae0c93c87c-dbus\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-etc-kubernetes\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719396 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2a21a5f2-be62-4282-96ee-616aaf35f34d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719883 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-lib-modules\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719894 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/056bd779-49dd-431f-89f8-0444aed7a025-iptables-alerter-script\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719910 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/056bd779-49dd-431f-89f8-0444aed7a025-host-slash\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.721145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n929q\" (UniqueName: \"kubernetes.io/projected/4c37b6ae-09dd-4587-a003-6ed07c747d40-kube-api-access-n929q\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8f2521-2a93-4462-89b6-e2d8aced60d5-tmp-dir\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719997 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-host-var-lib-kubelet\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720022 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-os-release\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720173 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-os-release\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.719963 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-run\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720185 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-systemd\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720247 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e9f16af0-c690-4e38-955a-2aae0c93c87c-dbus\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720295 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/009ea160-3568-48ea-bd35-ec7b42f5d1a9-etc-kubernetes\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720340 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c37b6ae-09dd-4587-a003-6ed07c747d40-lib-modules\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.720444 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:38.720494 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:14:39.220478038 +0000 UTC m=+2.069984380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720517 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8f2521-2a93-4462-89b6-e2d8aced60d5-tmp-dir\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.720916 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fe344e5a-2ae4-454a-8be3-bf168ef3ed1d-konnectivity-ca\") pod \"konnectivity-agent-fjqs9\" (UID: \"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d\") " pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.721027 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/009ea160-3568-48ea-bd35-ec7b42f5d1a9-cni-binary-copy\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.721665 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c37b6ae-09dd-4587-a003-6ed07c747d40-tmp\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.721786 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.721773 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c37b6ae-09dd-4587-a003-6ed07c747d40-etc-tuned\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.722780 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.722764 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fe344e5a-2ae4-454a-8be3-bf168ef3ed1d-agent-certs\") pod \"konnectivity-agent-fjqs9\" (UID: \"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d\") " pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.730292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.730265 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8dw7\" (UniqueName: \"kubernetes.io/projected/26b0b5ae-c523-4603-b958-bf5e8b6017cc-kube-api-access-d8dw7\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:38.730378 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.730361 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbdcm\" (UniqueName: \"kubernetes.io/projected/009ea160-3568-48ea-bd35-ec7b42f5d1a9-kube-api-access-rbdcm\") pod \"multus-zl4qh\" (UID: \"009ea160-3568-48ea-bd35-ec7b42f5d1a9\") " pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.734104 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.734077 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n929q\" (UniqueName: \"kubernetes.io/projected/4c37b6ae-09dd-4587-a003-6ed07c747d40-kube-api-access-n929q\") pod \"tuned-spdgx\" (UID: \"4c37b6ae-09dd-4587-a003-6ed07c747d40\") " pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.735669 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.735646 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kjpz\" (UniqueName: \"kubernetes.io/projected/5a8f2521-2a93-4462-89b6-e2d8aced60d5-kube-api-access-8kjpz\") pod \"node-resolver-7w5mz\" (UID: \"5a8f2521-2a93-4462-89b6-e2d8aced60d5\") " pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.737466 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.737446 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stms6\" (UniqueName: \"kubernetes.io/projected/2a21a5f2-be62-4282-96ee-616aaf35f34d-kube-api-access-stms6\") pod \"aws-ebs-csi-driver-node-2s4gb\" (UID: \"2a21a5f2-be62-4282-96ee-616aaf35f34d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.738252 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.738233 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k424t\" (UniqueName: \"kubernetes.io/projected/056bd779-49dd-431f-89f8-0444aed7a025-kube-api-access-k424t\") pod \"iptables-alerter-7bfv6\" (UID: \"056bd779-49dd-431f-89f8-0444aed7a025\") " pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.826281 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.826190 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:14:38.835398 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.835361 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c274373_f9bb_42eb_bc62_4b247a4df6f0.slice/crio-3b92a5d78dcccb461db8af2d070dc2d4a0859cfbc32e0c7c7dc8ee2c492f65d0 WatchSource:0}: Error finding container 3b92a5d78dcccb461db8af2d070dc2d4a0859cfbc32e0c7c7dc8ee2c492f65d0: Status 404 returned error can't find the container with id 3b92a5d78dcccb461db8af2d070dc2d4a0859cfbc32e0c7c7dc8ee2c492f65d0 Jul 16 17:14:38.849355 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.849324 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rv7rf" Jul 16 17:14:38.856992 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.856965 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd348bf_425d_45d0_8fe0_a91dfc261aae.slice/crio-8247754f494c13b09126fa62e6a0620a08e0520bd31e68c9b9277e43c9c2635e WatchSource:0}: Error finding container 8247754f494c13b09126fa62e6a0620a08e0520bd31e68c9b9277e43c9c2635e: Status 404 returned error can't find the container with id 8247754f494c13b09126fa62e6a0620a08e0520bd31e68c9b9277e43c9c2635e Jul 16 17:14:38.868143 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.868119 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t9r69" Jul 16 17:14:38.872884 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.872854 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7w5mz" Jul 16 17:14:38.877180 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.877147 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f4f3eb_183d_4fd1_99e6_af152bd9aca0.slice/crio-155bdc3917907aa79865e0d84ce92a8fbf25d96608c288524f0451abaa9967b2 WatchSource:0}: Error finding container 155bdc3917907aa79865e0d84ce92a8fbf25d96608c288524f0451abaa9967b2: Status 404 returned error can't find the container with id 155bdc3917907aa79865e0d84ce92a8fbf25d96608c288524f0451abaa9967b2 Jul 16 17:14:38.883089 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.883066 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8f2521_2a93_4462_89b6_e2d8aced60d5.slice/crio-f85ffbbeb90819e563a2b6b02b293e983c2e3c11f0ba629d3635225bc8d1758b WatchSource:0}: Error finding container f85ffbbeb90819e563a2b6b02b293e983c2e3c11f0ba629d3635225bc8d1758b: Status 404 returned error can't find the container with id f85ffbbeb90819e563a2b6b02b293e983c2e3c11f0ba629d3635225bc8d1758b Jul 16 17:14:38.883743 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.883719 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" Jul 16 17:14:38.891952 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.891930 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a21a5f2_be62_4282_96ee_616aaf35f34d.slice/crio-32a1049dccf99ca8745a3c48ca33f4cb151f9dff4029b8340a0fe717de88ad12 WatchSource:0}: Error finding container 32a1049dccf99ca8745a3c48ca33f4cb151f9dff4029b8340a0fe717de88ad12: Status 404 returned error can't find the container with id 32a1049dccf99ca8745a3c48ca33f4cb151f9dff4029b8340a0fe717de88ad12 Jul 16 17:14:38.897001 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.896983 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zl4qh" Jul 16 17:14:38.905511 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.905482 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009ea160_3568_48ea_bd35_ec7b42f5d1a9.slice/crio-90785ab9bc5dc9ee46b9d412754e40aa0dd3b22c4230577345e7c38043a1f3ed WatchSource:0}: Error finding container 90785ab9bc5dc9ee46b9d412754e40aa0dd3b22c4230577345e7c38043a1f3ed: Status 404 returned error can't find the container with id 90785ab9bc5dc9ee46b9d412754e40aa0dd3b22c4230577345e7c38043a1f3ed Jul 16 17:14:38.912357 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.912333 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7bfv6" Jul 16 17:14:38.919154 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.919134 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-spdgx" Jul 16 17:14:38.919425 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.919404 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056bd779_49dd_431f_89f8_0444aed7a025.slice/crio-4920dc7c627c979bb5d91533ecc31fd3f6114b2fec57d63e1616eaf994e555c4 WatchSource:0}: Error finding container 4920dc7c627c979bb5d91533ecc31fd3f6114b2fec57d63e1616eaf994e555c4: Status 404 returned error can't find the container with id 4920dc7c627c979bb5d91533ecc31fd3f6114b2fec57d63e1616eaf994e555c4 Jul 16 17:14:38.925809 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:38.925782 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:14:38.926165 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.926143 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c37b6ae_09dd_4587_a003_6ed07c747d40.slice/crio-a67754f90d617dce4d6a336c08f7aad987689b6332e6359d117697aa113eeacc WatchSource:0}: Error finding container a67754f90d617dce4d6a336c08f7aad987689b6332e6359d117697aa113eeacc: Status 404 returned error can't find the container with id a67754f90d617dce4d6a336c08f7aad987689b6332e6359d117697aa113eeacc Jul 16 17:14:38.933519 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:14:38.933498 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe344e5a_2ae4_454a_8be3_bf168ef3ed1d.slice/crio-6b0856ed286213b4c60f0891ca5f02471460f0f0bfe0d55d27efe608eacb6730 WatchSource:0}: Error finding container 6b0856ed286213b4c60f0891ca5f02471460f0f0bfe0d55d27efe608eacb6730: Status 404 returned error can't find the container with id 6b0856ed286213b4c60f0891ca5f02471460f0f0bfe0d55d27efe608eacb6730 Jul 16 17:14:39.224271 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.224168 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:39.224271 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.224250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:39.224498 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.224282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:39.224498 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224365 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:39.224498 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224441 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:39.224498 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224465 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:39.224498 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224479 2577 projected.go:196] Error preparing data for projected volume kube-api-access-zqfx7 for pod openshift-network-diagnostics/network-check-target-6cn29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:39.224770 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224591 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:39.224770 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224444 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:14:40.224422336 +0000 UTC m=+3.073928693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:39.224770 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224642 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7 podName:97fdadb3-57ac-4ab3-8f58-dfacdff7eca1 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:40.224623941 +0000 UTC m=+3.074130287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqfx7" (UniqueName: "kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7") pod "network-check-target-6cn29" (UID: "97fdadb3-57ac-4ab3-8f58-dfacdff7eca1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:39.224770 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:39.224659 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret podName:e9f16af0-c690-4e38-955a-2aae0c93c87c nodeName:}" failed. No retries permitted until 2026-07-16 17:14:40.224648036 +0000 UTC m=+3.074154383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret") pod "global-pull-secret-syncer-rp2zv" (UID: "e9f16af0-c690-4e38-955a-2aae0c93c87c") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:39.579767 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.579586 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 17:09:38 +0000 UTC" deadline="2028-02-22 11:22:23.076340062 +0000 UTC" Jul 16 17:14:39.579767 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.579628 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14058h7m43.496717905s" Jul 16 17:14:39.700986 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.700927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerStarted","Data":"155bdc3917907aa79865e0d84ce92a8fbf25d96608c288524f0451abaa9967b2"} Jul 16 17:14:39.713888 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.710184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"3b92a5d78dcccb461db8af2d070dc2d4a0859cfbc32e0c7c7dc8ee2c492f65d0"} Jul 16 17:14:39.715987 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.715882 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spdgx" event={"ID":"4c37b6ae-09dd-4587-a003-6ed07c747d40","Type":"ContainerStarted","Data":"a67754f90d617dce4d6a336c08f7aad987689b6332e6359d117697aa113eeacc"} Jul 16 17:14:39.721886 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.721827 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7bfv6" event={"ID":"056bd779-49dd-431f-89f8-0444aed7a025","Type":"ContainerStarted","Data":"4920dc7c627c979bb5d91533ecc31fd3f6114b2fec57d63e1616eaf994e555c4"} Jul 16 17:14:39.733811 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.733717 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zl4qh" event={"ID":"009ea160-3568-48ea-bd35-ec7b42f5d1a9","Type":"ContainerStarted","Data":"90785ab9bc5dc9ee46b9d412754e40aa0dd3b22c4230577345e7c38043a1f3ed"} Jul 16 17:14:39.742864 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.742763 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" event={"ID":"2a21a5f2-be62-4282-96ee-616aaf35f34d","Type":"ContainerStarted","Data":"32a1049dccf99ca8745a3c48ca33f4cb151f9dff4029b8340a0fe717de88ad12"} Jul 16 17:14:39.744832 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.744797 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rv7rf" event={"ID":"0bd348bf-425d-45d0-8fe0-a91dfc261aae","Type":"ContainerStarted","Data":"8247754f494c13b09126fa62e6a0620a08e0520bd31e68c9b9277e43c9c2635e"} Jul 16 17:14:39.750542 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.750444 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjqs9" event={"ID":"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d","Type":"ContainerStarted","Data":"6b0856ed286213b4c60f0891ca5f02471460f0f0bfe0d55d27efe608eacb6730"} Jul 16 17:14:39.759119 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.759091 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7w5mz" event={"ID":"5a8f2521-2a93-4462-89b6-e2d8aced60d5","Type":"ContainerStarted","Data":"f85ffbbeb90819e563a2b6b02b293e983c2e3c11f0ba629d3635225bc8d1758b"} Jul 16 17:14:39.982789 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:39.982696 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:40.234199 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.234106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:40.234199 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.234189 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:40.234439 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.234259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:40.234439 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.234400 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:40.234549 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.234478 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret podName:e9f16af0-c690-4e38-955a-2aae0c93c87c nodeName:}" failed. No retries permitted until 2026-07-16 17:14:42.234450523 +0000 UTC m=+5.083956885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret") pod "global-pull-secret-syncer-rp2zv" (UID: "e9f16af0-c690-4e38-955a-2aae0c93c87c") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:40.234969 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.234949 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:40.235058 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.234979 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:40.235058 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.235006 2577 projected.go:196] Error preparing data for projected volume kube-api-access-zqfx7 for pod openshift-network-diagnostics/network-check-target-6cn29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:40.235058 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.235053 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7 podName:97fdadb3-57ac-4ab3-8f58-dfacdff7eca1 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:42.235037586 +0000 UTC m=+5.084543942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqfx7" (UniqueName: "kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7") pod "network-check-target-6cn29" (UID: "97fdadb3-57ac-4ab3-8f58-dfacdff7eca1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:40.235226 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.235127 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:40.235226 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.235162 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:14:42.235150597 +0000 UTC m=+5.084656959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:40.580198 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.580069 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 17:09:38 +0000 UTC" deadline="2028-01-04 20:10:39.271864246 +0000 UTC" Jul 16 17:14:40.580198 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.580146 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12890h55m58.691723442s" Jul 16 17:14:40.662984 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.662952 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 17:14:40.681887 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.681854 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:40.682070 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.681990 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:40.682413 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.682391 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:40.682513 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.682491 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:40.682576 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:40.682567 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:40.682670 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:40.682650 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:42.251778 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:42.251737 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:42.251809 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:42.251866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.251936 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.251995 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.252034 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.252055 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.252058 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret podName:e9f16af0-c690-4e38-955a-2aae0c93c87c nodeName:}" failed. No retries permitted until 2026-07-16 17:14:46.252037977 +0000 UTC m=+9.101544321 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret") pod "global-pull-secret-syncer-rp2zv" (UID: "e9f16af0-c690-4e38-955a-2aae0c93c87c") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.252068 2577 projected.go:196] Error preparing data for projected volume kube-api-access-zqfx7 for pod openshift-network-diagnostics/network-check-target-6cn29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.252081 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:14:46.252070804 +0000 UTC m=+9.101577148 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:42.252235 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.252142 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7 podName:97fdadb3-57ac-4ab3-8f58-dfacdff7eca1 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:46.252126341 +0000 UTC m=+9.101632695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqfx7" (UniqueName: "kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7") pod "network-check-target-6cn29" (UID: "97fdadb3-57ac-4ab3-8f58-dfacdff7eca1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:42.681900 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:42.681817 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:42.681900 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:42.681853 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:42.682120 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:42.681946 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:42.682120 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.681964 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:42.682120 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.682046 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:42.682259 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:42.682138 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:44.681844 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:44.681802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:44.682280 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:44.681953 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:44.682280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:44.681802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:44.682280 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:44.682045 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:44.682280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:44.681802 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:44.682280 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:44.682131 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:46.285282 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:46.285229 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:46.285748 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:46.285296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:46.285748 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:46.285351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:46.285748 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.285472 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:46.285748 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.285533 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:14:54.28551464 +0000 UTC m=+17.135020991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:46.285996 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.285952 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:46.286049 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.286004 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret podName:e9f16af0-c690-4e38-955a-2aae0c93c87c nodeName:}" failed. No retries permitted until 2026-07-16 17:14:54.285990062 +0000 UTC m=+17.135496406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret") pod "global-pull-secret-syncer-rp2zv" (UID: "e9f16af0-c690-4e38-955a-2aae0c93c87c") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:46.286101 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.286079 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:46.286101 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.286093 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:46.286200 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.286105 2577 projected.go:196] Error preparing data for projected volume kube-api-access-zqfx7 for pod openshift-network-diagnostics/network-check-target-6cn29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:46.286200 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.286137 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7 podName:97fdadb3-57ac-4ab3-8f58-dfacdff7eca1 nodeName:}" failed. No retries permitted until 2026-07-16 17:14:54.286126223 +0000 UTC m=+17.135632582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqfx7" (UniqueName: "kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7") pod "network-check-target-6cn29" (UID: "97fdadb3-57ac-4ab3-8f58-dfacdff7eca1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:46.681681 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:46.681206 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:46.681681 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:46.681212 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:46.681681 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.681345 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:46.681681 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.681438 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:46.681681 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:46.681222 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:46.681681 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:46.681536 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:48.681871 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:48.681834 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:48.682386 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:48.681834 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:48.682386 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:48.681963 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:48.682386 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:48.681980 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:48.682386 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:48.682056 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:48.682386 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:48.682151 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:50.681807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:50.681762 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:50.681807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:50.681803 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:50.682284 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:50.681807 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:50.682284 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:50.681917 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:50.682284 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:50.682004 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:50.682284 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:50.682089 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:52.681338 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:52.681294 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:52.681338 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:52.681318 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:52.681338 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:52.681349 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:52.681872 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:52.681468 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:52.681872 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:52.681565 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:52.681872 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:52.681644 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:54.349472 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:54.349440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:54.349472 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:54.349479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:54.349514 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349602 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349620 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349641 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349652 2577 projected.go:196] Error preparing data for projected volume kube-api-access-zqfx7 for pod openshift-network-diagnostics/network-check-target-6cn29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349672 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret podName:e9f16af0-c690-4e38-955a-2aae0c93c87c nodeName:}" failed. No retries permitted until 2026-07-16 17:15:10.349650812 +0000 UTC m=+33.199157153 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret") pod "global-pull-secret-syncer-rp2zv" (UID: "e9f16af0-c690-4e38-955a-2aae0c93c87c") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349607 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349715 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7 podName:97fdadb3-57ac-4ab3-8f58-dfacdff7eca1 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:10.349699794 +0000 UTC m=+33.199206152 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqfx7" (UniqueName: "kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7") pod "network-check-target-6cn29" (UID: "97fdadb3-57ac-4ab3-8f58-dfacdff7eca1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:14:54.349989 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.349734 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:15:10.349722099 +0000 UTC m=+33.199228441 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:14:54.681321 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:54.681286 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:54.681321 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:54.681304 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:54.681559 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:54.681413 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:54.681559 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.681432 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:54.681559 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.681500 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:54.681706 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:54.681566 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:56.681858 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:56.681821 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:56.682357 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:56.681826 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:56.682357 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:56.681958 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:56.682357 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:56.681830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:56.682357 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:56.682037 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:56.682357 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:56.682102 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:58.681342 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:58.681164 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:14:58.681712 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:58.681164 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:14:58.681712 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:58.681433 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:14:58.681712 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:58.681178 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:14:58.681712 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:58.681492 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:14:58.681712 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:14:58.681586 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:14:59.801208 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.800994 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" event={"ID":"df2695e99c9c68ac5fa2b0530a286936","Type":"ContainerStarted","Data":"547964e78b6af067a43ce1658513f72230504bbb69562bd2a4faab05c72c7f1f"} Jul 16 17:14:59.802752 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.802523 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-spdgx" event={"ID":"4c37b6ae-09dd-4587-a003-6ed07c747d40","Type":"ContainerStarted","Data":"e78e4e95184ff9fa819fdb0b4e95b06485a21b6c6aa2c13f21aa2d85a6652d05"} Jul 16 17:14:59.803968 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.803946 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zl4qh" event={"ID":"009ea160-3568-48ea-bd35-ec7b42f5d1a9","Type":"ContainerStarted","Data":"75b72472cd39d7bec4ff233ab02df048c357798cb4a36d5ddec1ac8b77b05260"} Jul 16 17:14:59.805593 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.805567 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" event={"ID":"2a21a5f2-be62-4282-96ee-616aaf35f34d","Type":"ContainerStarted","Data":"f34f85e31cbeb295fc5499363ed448ea2920f0237a2b67727076613e99abda30"} Jul 16 17:14:59.807267 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.807247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rv7rf" event={"ID":"0bd348bf-425d-45d0-8fe0-a91dfc261aae","Type":"ContainerStarted","Data":"19f2426bc66b83abbe47412b4e04c6d0ff9892dc2eb69fde93b30c5f741e763d"} Jul 16 17:14:59.808678 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.808650 2577 generic.go:356] "Generic (PLEG): container finished" podID="d01a527a86fa60749d30ac5f9ca8fcf4" containerID="08759b161e7aad2abf0509860f64ef14104b68e761469f047784ea19a67f715d" exitCode=0 Jul 16 17:14:59.808774 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.808725 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" event={"ID":"d01a527a86fa60749d30ac5f9ca8fcf4","Type":"ContainerDied","Data":"08759b161e7aad2abf0509860f64ef14104b68e761469f047784ea19a67f715d"} Jul 16 17:14:59.812632 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.812607 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjqs9" event={"ID":"fe344e5a-2ae4-454a-8be3-bf168ef3ed1d","Type":"ContainerStarted","Data":"f46757a14c66cd4f203873699aa27c6d4992433cad0072197be4a52410f9f650"} Jul 16 17:14:59.813894 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.813871 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7w5mz" event={"ID":"5a8f2521-2a93-4462-89b6-e2d8aced60d5","Type":"ContainerStarted","Data":"a9fa6747b979efea2a4fc6b69e69204630bd58d289143c8579aa433d12aeee1d"} Jul 16 17:14:59.815298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.815276 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7f4f3eb-183d-4fd1-99e6-af152bd9aca0" containerID="c14dad1463e696c3075c2c75554e83439e13771aa9865b120cc301a512220ec6" exitCode=0 Jul 16 17:14:59.815392 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.815362 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerDied","Data":"c14dad1463e696c3075c2c75554e83439e13771aa9865b120cc301a512220ec6"} Jul 16 17:14:59.817447 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.817428 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:14:59.817739 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.817721 2577 generic.go:356] "Generic (PLEG): container finished" podID="3c274373-f9bb-42eb-bc62-4b247a4df6f0" containerID="9b1b849a0ae591e3774f4d50a9dcf400fe9a22938270815bc05dce2c9c6b0a40" exitCode=1 Jul 16 17:14:59.817834 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.817758 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"b003b14faffbfd056e6065bf32d431ddd09900cdbcdf5f097e0002c6a567b72b"} Jul 16 17:14:59.817913 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.817899 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"8408808d2542dbe51ac0dca07db34aec5813460ab6f1f49ccc16bcc989fb7038"} Jul 16 17:14:59.817944 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.817937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerDied","Data":"9b1b849a0ae591e3774f4d50a9dcf400fe9a22938270815bc05dce2c9c6b0a40"} Jul 16 17:14:59.817982 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.817975 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"930660faffe7cb4157cf9162272a6d7851e43ad3bf867f01766d526fbe606790"} Jul 16 17:14:59.820219 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.820174 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-160.ec2.internal" podStartSLOduration=21.820163449 podStartE2EDuration="21.820163449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:59.819658359 +0000 UTC m=+22.669164720" watchObservedRunningTime="2026-07-16 17:14:59.820163449 +0000 UTC m=+22.669669810" Jul 16 17:14:59.843560 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.843518 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7w5mz" podStartSLOduration=1.7245744699999999 podStartE2EDuration="21.843500919s" totalImagesPullingTime="20.118926449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:59.842894108 +0000 UTC m=+22.692400470" watchObservedRunningTime="2026-07-16 17:14:59.843500919 +0000 UTC m=+22.693007280" Jul 16 17:14:59.864358 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.864306 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zl4qh" podStartSLOduration=1.767549941 podStartE2EDuration="21.86428787s" totalImagesPullingTime="20.096737929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:59.864026527 +0000 UTC m=+22.713532889" watchObservedRunningTime="2026-07-16 17:14:59.86428787 +0000 UTC m=+22.713794232" Jul 16 17:14:59.884387 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.884341 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-spdgx" podStartSLOduration=1.891926948 podStartE2EDuration="21.884325147s" totalImagesPullingTime="19.992398199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:59.883909304 +0000 UTC m=+22.733415677" watchObservedRunningTime="2026-07-16 17:14:59.884325147 +0000 UTC m=+22.733831507" Jul 16 17:14:59.901557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.901513 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fjqs9" podStartSLOduration=3.130815101 podStartE2EDuration="21.901495644s" totalImagesPullingTime="18.770680543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:59.901098719 +0000 UTC m=+22.750605080" watchObservedRunningTime="2026-07-16 17:14:59.901495644 +0000 UTC m=+22.751002005" Jul 16 17:14:59.938817 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:14:59.938745 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rv7rf" podStartSLOduration=1.787984367 podStartE2EDuration="21.938724579s" totalImagesPullingTime="20.150740212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:14:59.938498635 +0000 UTC m=+22.788004996" watchObservedRunningTime="2026-07-16 17:14:59.938724579 +0000 UTC m=+22.788230941" Jul 16 17:15:00.042126 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.042084 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:00.115444 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.115412 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:00.681388 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.681349 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:00.681559 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.681400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:00.681559 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:00.681491 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:15:00.681658 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:00.681564 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:15:00.681658 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.681587 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:00.681771 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:00.681668 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:15:00.774322 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.774302 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 17:15:00.822621 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.822526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" event={"ID":"2a21a5f2-be62-4282-96ee-616aaf35f34d","Type":"ContainerStarted","Data":"3c7e280d4b460715ce4d0f3e4af8418b8d62f614ac5f7e8fea5b0b26bd80ac4a"} Jul 16 17:15:00.824563 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.824527 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" event={"ID":"d01a527a86fa60749d30ac5f9ca8fcf4","Type":"ContainerStarted","Data":"a789eca285df3b02cc803046d7966b3820e8b15d6fad3fb280bb57d058284fe0"} Jul 16 17:15:00.827659 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.827638 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:15:00.828087 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.828058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"e168a4b648a6d7925c116f0e5a12937d10ac7c43480af5447df0323182088fe5"} Jul 16 17:15:00.828219 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.828145 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"fa80fa546f407e927eb9b3b4e16c4f27a206ef083cd5a10251e86ba628d0c339"} Jul 16 17:15:00.829891 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.829855 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7bfv6" event={"ID":"056bd779-49dd-431f-89f8-0444aed7a025","Type":"ContainerStarted","Data":"1c4dc588d54252f52b933b96222c35a288fb2218c4b3e494b7e943354e3e2a0d"} Jul 16 17:15:00.845660 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.845609 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-160.ec2.internal" podStartSLOduration=22.845590546 podStartE2EDuration="22.845590546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:00.844045136 +0000 UTC m=+23.693551496" watchObservedRunningTime="2026-07-16 17:15:00.845590546 +0000 UTC m=+23.695096909" Jul 16 17:15:00.865141 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:00.865089 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7bfv6" podStartSLOduration=3.066008167 podStartE2EDuration="22.865070907s" totalImagesPullingTime="19.79906274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:00.864677113 +0000 UTC m=+23.714183474" watchObservedRunningTime="2026-07-16 17:15:00.865070907 +0000 UTC m=+23.714577268" Jul 16 17:15:01.142123 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:01.142096 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:01.218364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:01.218331 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:01.588143 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:01.587954 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T17:15:00.774318448Z","UUID":"d3b7dc4d-2565-4f9b-816e-507bc983810b","Handler":null,"Name":"","Endpoint":""} Jul 16 17:15:01.590232 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:01.590203 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 17:15:01.590389 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:01.590256 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 17:15:01.715645 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:01.715613 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:15:02.681213 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:02.681122 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:02.681213 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:02.681154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:02.681213 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:02.681154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:02.681802 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:02.681253 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:15:02.681802 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:02.681357 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:15:02.681802 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:02.681464 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:15:02.837315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:02.837269 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" event={"ID":"2a21a5f2-be62-4282-96ee-616aaf35f34d","Type":"ContainerStarted","Data":"fd9a18b70de03d326700860ffdc008f804a7a3aed46809b8bd137c310e3634c1"} Jul 16 17:15:02.840862 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:02.840835 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:15:02.841280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:02.841258 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"df165cac261ae248c975fd7c41fa371be5759fff321fbfe43c60e5b1fa6b040b"} Jul 16 17:15:02.863311 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:02.863247 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s4gb" podStartSLOduration=2.151348611 podStartE2EDuration="24.863225238s" totalImagesPullingTime="22.711876627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:02.862237899 +0000 UTC m=+25.711744261" watchObservedRunningTime="2026-07-16 17:15:02.863225238 +0000 UTC m=+25.712731602" Jul 16 17:15:04.406961 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.406672 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:15:04.407343 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.407287 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:15:04.681871 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.681832 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:04.681871 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.681858 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:04.682031 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.681832 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:04.682031 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:04.681945 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:15:04.682031 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:04.682000 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:15:04.682175 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:04.682070 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:15:04.847922 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.847833 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7f4f3eb-183d-4fd1-99e6-af152bd9aca0" containerID="c6df26667f7d585f7164538d76fb48e4d98bd70825b69adcd636fabf92d6a5fe" exitCode=0 Jul 16 17:15:04.848098 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.847921 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerDied","Data":"c6df26667f7d585f7164538d76fb48e4d98bd70825b69adcd636fabf92d6a5fe"} Jul 16 17:15:04.851399 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.851375 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:15:04.851824 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.851778 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"9d4030a2e3b27e7012e36b5519098f6cfe41b1b369900963dc40e812388cc82c"} Jul 16 17:15:04.852337 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.852319 2577 scope.go:117] "RemoveContainer" containerID="9b1b849a0ae591e3774f4d50a9dcf400fe9a22938270815bc05dce2c9c6b0a40" Jul 16 17:15:04.852506 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:04.852489 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fjqs9" Jul 16 17:15:05.001188 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.000989 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:05.861824 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.861634 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:15:05.863280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.863015 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" event={"ID":"3c274373-f9bb-42eb-bc62-4b247a4df6f0","Type":"ContainerStarted","Data":"519490a20b0b7772b1e8f2c725ebb121f6a70d049c88af71e09be2ad7bf511b2"} Jul 16 17:15:05.863280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.863088 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:15:05.863280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.863146 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:15:05.863280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.863248 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:15:05.895275 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.895207 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:15:05.895785 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.895669 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:15:05.903194 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:05.902428 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" podStartSLOduration=8.541649964 podStartE2EDuration="28.902408761s" totalImagesPullingTime="20.360758797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:05.90212617 +0000 UTC m=+28.751632552" watchObservedRunningTime="2026-07-16 17:15:05.902408761 +0000 UTC m=+28.751915125" Jul 16 17:15:06.681632 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:06.681603 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:06.681807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:06.681649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:06.681807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:06.681698 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:06.681807 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:06.681715 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:15:06.681931 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:06.681785 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:15:06.681931 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:06.681886 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:15:06.866377 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:06.866350 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7f4f3eb-183d-4fd1-99e6-af152bd9aca0" containerID="345eb0bc00bec107473e70f6c4b938c06030c4f7d2f6daabb34c5da9e288970b" exitCode=0 Jul 16 17:15:06.866714 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:06.866406 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerDied","Data":"345eb0bc00bec107473e70f6c4b938c06030c4f7d2f6daabb34c5da9e288970b"} Jul 16 17:15:07.016294 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:07.016224 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rp2zv"] Jul 16 17:15:07.016412 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:07.016318 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:07.016445 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:07.016402 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:15:07.022939 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:07.022808 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6cn29"] Jul 16 17:15:07.023095 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:07.022980 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:07.023196 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:07.023139 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:15:07.023350 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:07.023329 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-27zcc"] Jul 16 17:15:07.023473 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:07.023460 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:07.023934 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:07.023905 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:15:08.681472 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:08.681296 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:08.681895 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:08.681322 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:08.681895 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:08.681546 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:15:08.681895 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:08.681612 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:15:08.681895 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:08.681327 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:08.681895 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:08.681758 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:15:08.873294 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:08.873253 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7f4f3eb-183d-4fd1-99e6-af152bd9aca0" containerID="cd952b5a4ab6d8c55d4d13975ccc941f971233d66f274618c2dba99e8c4ec586" exitCode=0 Jul 16 17:15:08.873445 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:08.873305 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerDied","Data":"cd952b5a4ab6d8c55d4d13975ccc941f971233d66f274618c2dba99e8c4ec586"} Jul 16 17:15:10.373169 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:10.373119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:10.373169 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:10.373175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:10.373216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373270 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373331 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373337 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373355 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373364 2577 projected.go:196] Error preparing data for projected volume kube-api-access-zqfx7 for pod openshift-network-diagnostics/network-check-target-6cn29: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373335 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret podName:e9f16af0-c690-4e38-955a-2aae0c93c87c nodeName:}" failed. No retries permitted until 2026-07-16 17:15:42.373319548 +0000 UTC m=+65.222825886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret") pod "global-pull-secret-syncer-rp2zv" (UID: "e9f16af0-c690-4e38-955a-2aae0c93c87c") : object "kube-system"/"original-pull-secret" not registered Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373391 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:15:42.373375929 +0000 UTC m=+65.222882268 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 17:15:10.373655 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.373406 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7 podName:97fdadb3-57ac-4ab3-8f58-dfacdff7eca1 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:42.373397618 +0000 UTC m=+65.222903961 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zqfx7" (UniqueName: "kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7") pod "network-check-target-6cn29" (UID: "97fdadb3-57ac-4ab3-8f58-dfacdff7eca1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 17:15:10.681976 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:10.681943 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:10.681976 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:10.681978 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:10.682174 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:10.682050 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:10.682174 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.682065 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6cn29" podUID="97fdadb3-57ac-4ab3-8f58-dfacdff7eca1" Jul 16 17:15:10.682174 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.682119 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rp2zv" podUID="e9f16af0-c690-4e38-955a-2aae0c93c87c" Jul 16 17:15:10.682260 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:10.682190 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-27zcc" podUID="26b0b5ae-c523-4603-b958-bf5e8b6017cc" Jul 16 17:15:12.010349 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.010321 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-160.ec2.internal" event="NodeReady" Jul 16 17:15:12.010871 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.010432 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 17:15:12.071954 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.071924 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4h2x5"] Jul 16 17:15:12.073952 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.073937 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.077119 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.077094 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 17:15:12.077236 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.077131 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bfmt5\"" Jul 16 17:15:12.077236 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.077222 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 17:15:12.077637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.077619 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fc7ll"] Jul 16 17:15:12.079303 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.079284 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.082453 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.082426 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gn8hz\"" Jul 16 17:15:12.082453 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.082436 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 17:15:12.082670 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.082655 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 17:15:12.082740 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.082676 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 17:15:12.093063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.093044 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4h2x5"] Jul 16 17:15:12.110841 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.110815 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fc7ll"] Jul 16 17:15:12.186840 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.186805 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-tmp-dir\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.187000 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.186873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a5c642e-79dd-419a-b4d0-c309f7988d04-cert\") pod \"ingress-canary-fc7ll\" (UID: \"5a5c642e-79dd-419a-b4d0-c309f7988d04\") " pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.187000 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.186915 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmpsl\" (UniqueName: \"kubernetes.io/projected/5a5c642e-79dd-419a-b4d0-c309f7988d04-kube-api-access-gmpsl\") pod \"ingress-canary-fc7ll\" (UID: \"5a5c642e-79dd-419a-b4d0-c309f7988d04\") " pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.187000 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.186944 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-config-volume\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.187141 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.186999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7srs\" (UniqueName: \"kubernetes.io/projected/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-kube-api-access-r7srs\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.187141 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.187062 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.287705 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.287622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7srs\" (UniqueName: \"kubernetes.io/projected/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-kube-api-access-r7srs\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.287705 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.287678 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.287953 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.287712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-tmp-dir\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.287953 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.287741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a5c642e-79dd-419a-b4d0-c309f7988d04-cert\") pod \"ingress-canary-fc7ll\" (UID: \"5a5c642e-79dd-419a-b4d0-c309f7988d04\") " pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.287953 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.287757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmpsl\" (UniqueName: \"kubernetes.io/projected/5a5c642e-79dd-419a-b4d0-c309f7988d04-kube-api-access-gmpsl\") pod \"ingress-canary-fc7ll\" (UID: \"5a5c642e-79dd-419a-b4d0-c309f7988d04\") " pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.287953 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.287779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-config-volume\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.287953 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:12.287857 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:12.287953 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:12.287917 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:15:12.787898475 +0000 UTC m=+35.637404816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:15:12.288210 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.288117 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-tmp-dir\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.288354 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.288331 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-config-volume\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.292002 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.291984 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a5c642e-79dd-419a-b4d0-c309f7988d04-cert\") pod \"ingress-canary-fc7ll\" (UID: \"5a5c642e-79dd-419a-b4d0-c309f7988d04\") " pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.312033 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.312007 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7srs\" (UniqueName: \"kubernetes.io/projected/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-kube-api-access-r7srs\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.312712 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.312693 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmpsl\" (UniqueName: \"kubernetes.io/projected/5a5c642e-79dd-419a-b4d0-c309f7988d04-kube-api-access-gmpsl\") pod \"ingress-canary-fc7ll\" (UID: \"5a5c642e-79dd-419a-b4d0-c309f7988d04\") " pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.337313 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.337288 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:12.388360 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.388329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fc7ll" Jul 16 17:15:12.443228 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.443200 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:12.528400 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.528192 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fc7ll"] Jul 16 17:15:12.532280 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:15:12.532257 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5c642e_79dd_419a_b4d0_c309f7988d04.slice/crio-dbb41f22692afdb6b7ec6ad4a28d6c2c520953aad6fcef6decc0f33871cb9f03 WatchSource:0}: Error finding container dbb41f22692afdb6b7ec6ad4a28d6c2c520953aad6fcef6decc0f33871cb9f03: Status 404 returned error can't find the container with id dbb41f22692afdb6b7ec6ad4a28d6c2c520953aad6fcef6decc0f33871cb9f03 Jul 16 17:15:12.681807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.681746 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:12.681807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.681764 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:12.681807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.681760 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:12.685145 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.685124 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 17:15:12.686368 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.686341 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-65bh8\"" Jul 16 17:15:12.686464 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.686397 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 17:15:12.686464 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.686440 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9l5d5\"" Jul 16 17:15:12.686464 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.686446 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 17:15:12.686594 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.686579 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 17:15:12.791919 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.791882 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:12.792082 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:12.792005 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:12.792082 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:12.792068 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:15:13.792049615 +0000 UTC m=+36.641555957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:15:12.882843 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:12.882807 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fc7ll" event={"ID":"5a5c642e-79dd-419a-b4d0-c309f7988d04","Type":"ContainerStarted","Data":"dbb41f22692afdb6b7ec6ad4a28d6c2c520953aad6fcef6decc0f33871cb9f03"} Jul 16 17:15:13.799887 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:13.799848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:13.800510 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:13.800039 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:13.800510 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:13.800105 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:15:15.800083042 +0000 UTC m=+38.649589381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:15:15.816004 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:15.815963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:15.816428 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:15.816121 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:15.816428 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:15.816194 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:15:19.816175553 +0000 UTC m=+42.665681900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:15:16.532465 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:16.532433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:16.599544 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:16.599511 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:16.900719 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:16.900658 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:16.901215 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:16.901136 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerStarted","Data":"97e07e40eaae380c31bc25b310de7ffcedcc4afe61f899e746c0a37850ec1769"} Jul 16 17:15:16.902861 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:16.902829 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fc7ll" event={"ID":"5a5c642e-79dd-419a-b4d0-c309f7988d04","Type":"ContainerStarted","Data":"4ab57fa09b1f38ca3492acd5e937b6504bd3d8edbf0666cb2812cfae098ac137"} Jul 16 17:15:16.953416 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:16.953366 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fc7ll" podStartSLOduration=0.759175223 podStartE2EDuration="4.953347864s" totalImagesPullingTime="4.194172641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:16.952746773 +0000 UTC m=+39.802253133" watchObservedRunningTime="2026-07-16 17:15:16.953347864 +0000 UTC m=+39.802854223" Jul 16 17:15:16.994711 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:16.994642 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:17.908065 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:17.908028 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7f4f3eb-183d-4fd1-99e6-af152bd9aca0" containerID="97e07e40eaae380c31bc25b310de7ffcedcc4afe61f899e746c0a37850ec1769" exitCode=0 Jul 16 17:15:17.908431 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:17.908104 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerDied","Data":"97e07e40eaae380c31bc25b310de7ffcedcc4afe61f899e746c0a37850ec1769"} Jul 16 17:15:18.913725 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:18.913694 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7f4f3eb-183d-4fd1-99e6-af152bd9aca0" containerID="00699c0f38bc770ae51a8684715e21c8549e4b9b08d334919ed401d23bb49ba0" exitCode=0 Jul 16 17:15:18.914112 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:18.913746 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerDied","Data":"00699c0f38bc770ae51a8684715e21c8549e4b9b08d334919ed401d23bb49ba0"} Jul 16 17:15:19.846891 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:19.846853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:19.847035 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:19.846965 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:19.847035 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:19.847017 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:15:27.847003467 +0000 UTC m=+50.696509805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:15:19.919488 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:19.919450 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t9r69" event={"ID":"b7f4f3eb-183d-4fd1-99e6-af152bd9aca0","Type":"ContainerStarted","Data":"d95cb9d11f80f7c016ea5ffd878dd17cae4ec18bd1bdc18642571b55198c2d89"} Jul 16 17:15:19.955053 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:19.955010 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t9r69" podStartSLOduration=7.469781477 podStartE2EDuration="41.954995835s" totalImagesPullingTime="34.485214358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:19.953343307 +0000 UTC m=+42.802849666" watchObservedRunningTime="2026-07-16 17:15:19.954995835 +0000 UTC m=+42.804502208" Jul 16 17:15:21.722227 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:21.722192 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:27.906479 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:27.906446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:27.906908 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:27.906571 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:27.906908 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:27.906627 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:15:43.906614799 +0000 UTC m=+66.756121137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:15:37.885785 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:37.885755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wsjz" Jul 16 17:15:38.061927 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:38.061890 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:38.133360 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:38.133332 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:42.400879 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.400823 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:42.400879 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.400881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:42.401347 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.400917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:15:42.407150 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.407132 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 17:15:42.407259 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.407243 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 17:15:42.407302 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.407295 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 17:15:42.411384 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:42.411369 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 17:15:42.411438 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:42.411427 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs podName:26b0b5ae-c523-4603-b958-bf5e8b6017cc nodeName:}" failed. No retries permitted until 2026-07-16 17:16:46.411413937 +0000 UTC m=+129.260920276 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs") pod "network-metrics-daemon-27zcc" (UID: "26b0b5ae-c523-4603-b958-bf5e8b6017cc") : secret "metrics-daemon-secret" not found Jul 16 17:15:42.414362 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.414343 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e9f16af0-c690-4e38-955a-2aae0c93c87c-original-pull-secret\") pod \"global-pull-secret-syncer-rp2zv\" (UID: \"e9f16af0-c690-4e38-955a-2aae0c93c87c\") " pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:42.415960 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.415941 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 17:15:42.423530 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.423509 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqfx7\" (UniqueName: \"kubernetes.io/projected/97fdadb3-57ac-4ab3-8f58-dfacdff7eca1-kube-api-access-zqfx7\") pod \"network-check-target-6cn29\" (UID: \"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1\") " pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:42.693977 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.693884 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rp2zv" Jul 16 17:15:42.703617 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.703588 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9l5d5\"" Jul 16 17:15:42.711491 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.711469 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:42.830081 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.830039 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rp2zv"] Jul 16 17:15:42.834014 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:15:42.833990 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f16af0_c690_4e38_955a_2aae0c93c87c.slice/crio-3d68bf21cf16035a6a1bfd0c8b93bc0209bbf1c098282b01366dc6132cb438d4 WatchSource:0}: Error finding container 3d68bf21cf16035a6a1bfd0c8b93bc0209bbf1c098282b01366dc6132cb438d4: Status 404 returned error can't find the container with id 3d68bf21cf16035a6a1bfd0c8b93bc0209bbf1c098282b01366dc6132cb438d4 Jul 16 17:15:42.847004 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.846981 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6cn29"] Jul 16 17:15:42.849442 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:15:42.849417 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97fdadb3_57ac_4ab3_8f58_dfacdff7eca1.slice/crio-6293c8dd9d2740ef5301b9c184d1148f410b2fc94920f056f4620451083278ae WatchSource:0}: Error finding container 6293c8dd9d2740ef5301b9c184d1148f410b2fc94920f056f4620451083278ae: Status 404 returned error can't find the container with id 6293c8dd9d2740ef5301b9c184d1148f410b2fc94920f056f4620451083278ae Jul 16 17:15:42.970931 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.970852 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6cn29" event={"ID":"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1","Type":"ContainerStarted","Data":"6293c8dd9d2740ef5301b9c184d1148f410b2fc94920f056f4620451083278ae"} Jul 16 17:15:42.971922 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:42.971902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rp2zv" event={"ID":"e9f16af0-c690-4e38-955a-2aae0c93c87c","Type":"ContainerStarted","Data":"3d68bf21cf16035a6a1bfd0c8b93bc0209bbf1c098282b01366dc6132cb438d4"} Jul 16 17:15:43.912446 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:43.912352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:15:43.913039 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:43.912517 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:15:43.913039 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:43.912590 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:16:15.912570615 +0000 UTC m=+98.762076956 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:15:46.984138 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:46.984106 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6cn29" event={"ID":"97fdadb3-57ac-4ab3-8f58-dfacdff7eca1","Type":"ContainerStarted","Data":"2a569ab20698a089c2c321ef31374f14ac6dbc7aab5700d772a6106f26cdd2aa"} Jul 16 17:15:46.984494 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:46.984198 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:15:47.005866 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:47.005822 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6cn29" podStartSLOduration=65.937920341 podStartE2EDuration="1m10.005806449s" totalImagesPullingTime="4.067886108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:47.004409732 +0000 UTC m=+69.853916090" watchObservedRunningTime="2026-07-16 17:15:47.005806449 +0000 UTC m=+69.855312807" Jul 16 17:15:47.987600 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:47.987560 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rp2zv" event={"ID":"e9f16af0-c690-4e38-955a-2aae0c93c87c","Type":"ContainerStarted","Data":"2602f5d67483524dba7f67a14328d6f189161a30ce32f183f0563a2c36d9f90d"} Jul 16 17:15:48.007957 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:48.007911 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rp2zv" podStartSLOduration=65.921394742 podStartE2EDuration="1m10.007897715s" totalImagesPullingTime="4.086502973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:15:48.007712514 +0000 UTC m=+70.857218873" watchObservedRunningTime="2026-07-16 17:15:48.007897715 +0000 UTC m=+70.857404053" Jul 16 17:15:56.185383 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.185345 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dwrfw"] Jul 16 17:15:56.187994 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.187973 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.190584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.190565 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 17:15:56.196538 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.196523 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 17:15:56.197637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.197613 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 17:15:56.197720 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.197615 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 17:15:56.197770 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.197718 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nzkrg\"" Jul 16 17:15:56.203144 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.203131 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 17:15:56.216903 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.216884 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dwrfw"] Jul 16 17:15:56.287686 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.287657 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53f52dc0-37b5-4f63-814c-599b8fd74feb-tmp\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.287686 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.287690 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53f52dc0-37b5-4f63-814c-599b8fd74feb-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.287875 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.287707 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53f52dc0-37b5-4f63-814c-599b8fd74feb-serving-cert\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.287875 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.287777 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53f52dc0-37b5-4f63-814c-599b8fd74feb-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.287875 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.287829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/53f52dc0-37b5-4f63-814c-599b8fd74feb-snapshots\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.287875 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.287850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44mf9\" (UniqueName: \"kubernetes.io/projected/53f52dc0-37b5-4f63-814c-599b8fd74feb-kube-api-access-44mf9\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.288543 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.288525 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-568468f9f4-9h48n"] Jul 16 17:15:56.290284 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.290269 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.293402 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.293384 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 17:15:56.293525 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.293509 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 17:15:56.293594 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.293546 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 17:15:56.293669 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.293654 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gnm25\"" Jul 16 17:15:56.294305 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.294290 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 17:15:56.294391 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.294328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 17:15:56.294767 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.294748 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 17:15:56.307136 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.307115 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-568468f9f4-9h48n"] Jul 16 17:15:56.388513 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/53f52dc0-37b5-4f63-814c-599b8fd74feb-snapshots\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.388513 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-default-certificate\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.388673 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-stats-auth\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.388673 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388550 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44mf9\" (UniqueName: \"kubernetes.io/projected/53f52dc0-37b5-4f63-814c-599b8fd74feb-kube-api-access-44mf9\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.388673 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.388673 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388632 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f69d9\" (UniqueName: \"kubernetes.io/projected/9790177d-d4dc-417f-aa42-f714a625d827-kube-api-access-f69d9\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.388844 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388688 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53f52dc0-37b5-4f63-814c-599b8fd74feb-tmp\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.388844 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53f52dc0-37b5-4f63-814c-599b8fd74feb-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.388844 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53f52dc0-37b5-4f63-814c-599b8fd74feb-serving-cert\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.389000 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.388980 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.389045 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.389024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53f52dc0-37b5-4f63-814c-599b8fd74feb-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.389215 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.389197 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53f52dc0-37b5-4f63-814c-599b8fd74feb-tmp\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.389296 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.389223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/53f52dc0-37b5-4f63-814c-599b8fd74feb-snapshots\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.389334 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.389301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53f52dc0-37b5-4f63-814c-599b8fd74feb-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.389751 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.389733 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53f52dc0-37b5-4f63-814c-599b8fd74feb-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.390909 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.390894 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53f52dc0-37b5-4f63-814c-599b8fd74feb-serving-cert\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.404987 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.404967 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44mf9\" (UniqueName: \"kubernetes.io/projected/53f52dc0-37b5-4f63-814c-599b8fd74feb-kube-api-access-44mf9\") pod \"insights-operator-68d9dd94cf-dwrfw\" (UID: \"53f52dc0-37b5-4f63-814c-599b8fd74feb\") " pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.490054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.489982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.490054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.490017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-default-certificate\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.490054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.490033 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-stats-auth\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.490054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.490051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.490331 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.490084 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f69d9\" (UniqueName: \"kubernetes.io/projected/9790177d-d4dc-417f-aa42-f714a625d827-kube-api-access-f69d9\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.490331 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:56.490164 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:56.990142344 +0000 UTC m=+79.839648682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : configmap references non-existent config key: service-ca.crt Jul 16 17:15:56.490331 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:56.490174 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 17:15:56.490331 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:56.490221 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:56.990209675 +0000 UTC m=+79.839716016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : secret "router-metrics-certs-default" not found Jul 16 17:15:56.492275 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.492258 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-stats-auth\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.492386 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.492370 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-default-certificate\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.496200 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.496180 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" Jul 16 17:15:56.509504 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.509476 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f69d9\" (UniqueName: \"kubernetes.io/projected/9790177d-d4dc-417f-aa42-f714a625d827-kube-api-access-f69d9\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.528081 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.528054 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:56.615236 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.615201 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-dwrfw"] Jul 16 17:15:56.620244 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:15:56.620205 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f52dc0_37b5_4f63_814c_599b8fd74feb.slice/crio-65e4c5101605116924ec867e62c86e2fdb52fa42866c34700127f7e34a75685b WatchSource:0}: Error finding container 65e4c5101605116924ec867e62c86e2fdb52fa42866c34700127f7e34a75685b: Status 404 returned error can't find the container with id 65e4c5101605116924ec867e62c86e2fdb52fa42866c34700127f7e34a75685b Jul 16 17:15:56.621118 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.621095 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:15:56.994462 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.994436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.994631 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:56.994497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:56.994631 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:56.994581 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 17:15:56.994631 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:56.994606 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:57.994591755 +0000 UTC m=+80.844098093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : configmap references non-existent config key: service-ca.crt Jul 16 17:15:56.994631 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:56.994630 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:15:57.994617428 +0000 UTC m=+80.844123768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : secret "router-metrics-certs-default" not found Jul 16 17:15:57.008667 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:57.008641 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" event={"ID":"53f52dc0-37b5-4f63-814c-599b8fd74feb","Type":"ContainerStarted","Data":"65e4c5101605116924ec867e62c86e2fdb52fa42866c34700127f7e34a75685b"} Jul 16 17:15:58.000742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:58.000700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:58.000742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:15:58.000755 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:15:58.001190 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:58.000865 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 17:15:58.001190 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:58.000907 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:00.000893485 +0000 UTC m=+82.850399824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : secret "router-metrics-certs-default" not found Jul 16 17:15:58.001190 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:15:58.000920 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:00.000912725 +0000 UTC m=+82.850419064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : configmap references non-existent config key: service-ca.crt Jul 16 17:16:00.014737 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:00.014634 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:00.014737 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:00.014681 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:00.015150 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:00.014817 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:04.014783141 +0000 UTC m=+86.864289484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : configmap references non-existent config key: service-ca.crt Jul 16 17:16:00.015150 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:00.014817 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 17:16:00.015150 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:00.014861 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:04.014853469 +0000 UTC m=+86.864359808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : secret "router-metrics-certs-default" not found Jul 16 17:16:04.044534 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:04.044491 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:04.045117 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:04.044676 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:04.045117 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:04.044678 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 17:16:04.045117 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:04.044802 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:12.044767883 +0000 UTC m=+94.894274222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : configmap references non-existent config key: service-ca.crt Jul 16 17:16:04.045117 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:04.044829 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:12.044817625 +0000 UTC m=+94.894323965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : secret "router-metrics-certs-default" not found Jul 16 17:16:05.028007 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.027974 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" event={"ID":"53f52dc0-37b5-4f63-814c-599b8fd74feb","Type":"ContainerStarted","Data":"290c60649acfe70764533ab4a109aee98bcab136b4e893cf34a7930fb192a2b9"} Jul 16 17:16:05.055615 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.055566 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" podStartSLOduration=1.179827476 podStartE2EDuration="9.055550946s" totalImagesPullingTime="7.87572347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:05.053622 +0000 UTC m=+87.903128362" watchObservedRunningTime="2026-07-16 17:16:05.055550946 +0000 UTC m=+87.905057307" Jul 16 17:16:05.821864 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.821834 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q9qkd"] Jul 16 17:16:05.823785 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.823769 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.829860 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.829545 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 17:16:05.831372 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.831354 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 17:16:05.834658 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.834637 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t9pcw\"" Jul 16 17:16:05.855491 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.855458 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q9qkd"] Jul 16 17:16:05.858808 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.858757 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32c35b6-d851-4c34-b4b6-5586ca0e2357-crio-socket\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.858931 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.858844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.858931 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.858871 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs6br\" (UniqueName: \"kubernetes.io/projected/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-api-access-vs6br\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.859036 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.858950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.859036 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.859012 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32c35b6-d851-4c34-b4b6-5586ca0e2357-data-volume\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.873328 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.873298 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:05.958017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.957976 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:05.960091 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.960067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.960184 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.960111 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32c35b6-d851-4c34-b4b6-5586ca0e2357-data-volume\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.960184 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.960162 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32c35b6-d851-4c34-b4b6-5586ca0e2357-crio-socket\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.960288 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:05.960186 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:05.960288 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:05.960239 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:06.460223536 +0000 UTC m=+89.309729879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : configmap "kube-rbac-proxy" not found Jul 16 17:16:05.960288 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.960188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.960288 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.960267 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs6br\" (UniqueName: \"kubernetes.io/projected/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-api-access-vs6br\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.960288 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:05.960276 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:05.960612 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.960292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f32c35b6-d851-4c34-b4b6-5586ca0e2357-crio-socket\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.960612 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:05.960332 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:06.460318308 +0000 UTC m=+89.309824651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:05.960612 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.960473 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f32c35b6-d851-4c34-b4b6-5586ca0e2357-data-volume\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:05.975962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:05.975934 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs6br\" (UniqueName: \"kubernetes.io/projected/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-api-access-vs6br\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:06.464547 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:06.464510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:06.464930 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:06.464558 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:06.464930 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:06.464622 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:06.464930 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:06.464662 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:06.464930 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:06.464672 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:07.464659075 +0000 UTC m=+90.314165414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : configmap "kube-rbac-proxy" not found Jul 16 17:16:06.464930 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:06.464726 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:07.464710713 +0000 UTC m=+90.314217051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:07.472919 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:07.472875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:07.472919 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:07.472933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:07.473341 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:07.473022 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:07.473341 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:07.473039 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:07.473341 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:07.473074 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:09.473060789 +0000 UTC m=+92.322567128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : configmap "kube-rbac-proxy" not found Jul 16 17:16:07.473341 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:07.473092 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:09.473079928 +0000 UTC m=+92.322586268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:07.581378 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:07.581343 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7w5mz_5a8f2521-2a93-4462-89b6-e2d8aced60d5/dns-node-resolver/0.log" Jul 16 17:16:08.765933 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:08.765904 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rv7rf_0bd348bf-425d-45d0-8fe0-a91dfc261aae/node-ca/0.log" Jul 16 17:16:08.766285 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:08.766165 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:09.361992 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:09.361963 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fc7ll_5a5c642e-79dd-419a-b4d0-c309f7988d04/serve-healthcheck-canary/0.log" Jul 16 17:16:09.490766 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:09.490720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:09.490766 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:09.490777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:09.491017 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:09.490877 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:09.491017 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:09.490880 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:09.491017 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:09.490941 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:13.490926134 +0000 UTC m=+96.340432474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:09.491017 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:09.490958 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:13.490950334 +0000 UTC m=+96.340456673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : configmap "kube-rbac-proxy" not found Jul 16 17:16:11.005696 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.005666 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:11.051521 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.051491 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp"] Jul 16 17:16:11.053647 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.053630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" Jul 16 17:16:11.061925 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.061905 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-8r4rc\"" Jul 16 17:16:11.062175 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.062159 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 16 17:16:11.063064 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.063052 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 16 17:16:11.080722 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.080695 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp"] Jul 16 17:16:11.104628 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.104601 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2g82\" (UniqueName: \"kubernetes.io/projected/efae4550-d04d-49c5-9cee-f54b06131155-kube-api-access-m2g82\") pod \"migrator-7f59b54f6c-klqmp\" (UID: \"efae4550-d04d-49c5-9cee-f54b06131155\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" Jul 16 17:16:11.112620 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.112593 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:11.197389 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.197324 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:11.205421 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.205389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2g82\" (UniqueName: \"kubernetes.io/projected/efae4550-d04d-49c5-9cee-f54b06131155-kube-api-access-m2g82\") pod \"migrator-7f59b54f6c-klqmp\" (UID: \"efae4550-d04d-49c5-9cee-f54b06131155\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" Jul 16 17:16:11.217806 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.217776 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2g82\" (UniqueName: \"kubernetes.io/projected/efae4550-d04d-49c5-9cee-f54b06131155-kube-api-access-m2g82\") pod \"migrator-7f59b54f6c-klqmp\" (UID: \"efae4550-d04d-49c5-9cee-f54b06131155\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" Jul 16 17:16:11.362251 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.362168 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" Jul 16 17:16:11.481973 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:11.481945 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp"] Jul 16 17:16:11.484901 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:11.484874 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefae4550_d04d_49c5_9cee_f54b06131155.slice/crio-8ad5a7fab1047fa5c065d949f353e81ca13eba38775361c9149d538301dc7e83 WatchSource:0}: Error finding container 8ad5a7fab1047fa5c065d949f353e81ca13eba38775361c9149d538301dc7e83: Status 404 returned error can't find the container with id 8ad5a7fab1047fa5c065d949f353e81ca13eba38775361c9149d538301dc7e83 Jul 16 17:16:12.046254 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:12.046220 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" event={"ID":"efae4550-d04d-49c5-9cee-f54b06131155","Type":"ContainerStarted","Data":"8ad5a7fab1047fa5c065d949f353e81ca13eba38775361c9149d538301dc7e83"} Jul 16 17:16:12.112356 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:12.112321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:12.112524 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:12.112410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:12.112524 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:12.112462 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 17:16:12.112599 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:12.112537 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:28.112521563 +0000 UTC m=+110.962027902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : secret "router-metrics-certs-default" not found Jul 16 17:16:12.112599 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:12.112552 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle podName:9790177d-d4dc-417f-aa42-f714a625d827 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:28.112546048 +0000 UTC m=+110.962052387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle") pod "router-default-568468f9f4-9h48n" (UID: "9790177d-d4dc-417f-aa42-f714a625d827") : configmap references non-existent config key: service-ca.crt Jul 16 17:16:12.341228 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:12.341151 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:12.412703 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:12.412667 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:13.050947 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:13.050904 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" event={"ID":"efae4550-d04d-49c5-9cee-f54b06131155","Type":"ContainerStarted","Data":"bf90e54883b8e24fdf79dbbd6962e619b66fe5e2a549de5a26ca6f9eb9d50162"} Jul 16 17:16:13.051332 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:13.050994 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" event={"ID":"efae4550-d04d-49c5-9cee-f54b06131155","Type":"ContainerStarted","Data":"7b46a6e547aca4c86aed62958dbba5bb4a19aae816123a30ca4d7602c257cedd"} Jul 16 17:16:13.074864 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:13.074809 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-klqmp" podStartSLOduration=1.022052108 podStartE2EDuration="2.074775253s" totalImagesPullingTime="1.052723145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:13.073518125 +0000 UTC m=+95.923024487" watchObservedRunningTime="2026-07-16 17:16:13.074775253 +0000 UTC m=+95.924281614" Jul 16 17:16:13.510701 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:13.510668 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:13.524801 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:13.524767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:13.524893 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:13.524871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:13.524957 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:13.524895 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 17:16:13.525013 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:13.524957 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:13.525013 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:13.524977 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:21.524952001 +0000 UTC m=+104.374458354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : configmap "kube-rbac-proxy" not found Jul 16 17:16:13.525013 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:13.525004 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:21.524992855 +0000 UTC m=+104.374499193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:13.608839 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:13.608814 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:14.678823 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.678774 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8mpmw"] Jul 16 17:16:14.680777 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.680761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.684038 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.684018 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 16 17:16:14.685255 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.685234 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 16 17:16:14.685383 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.685257 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tr24j\"" Jul 16 17:16:14.685383 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.685289 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 16 17:16:14.685383 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.685259 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 16 17:16:14.696227 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.696195 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8mpmw"] Jul 16 17:16:14.723592 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.723564 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:14.733635 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.733611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/03c047d0-92c4-43dc-8825-53ee34685725-signing-key\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.733730 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.733714 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/03c047d0-92c4-43dc-8825-53ee34685725-signing-cabundle\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.733828 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.733779 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jkgp\" (UniqueName: \"kubernetes.io/projected/03c047d0-92c4-43dc-8825-53ee34685725-kube-api-access-6jkgp\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.801860 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.801838 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:14.834364 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.834343 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/03c047d0-92c4-43dc-8825-53ee34685725-signing-cabundle\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.834444 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.834403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jkgp\" (UniqueName: \"kubernetes.io/projected/03c047d0-92c4-43dc-8825-53ee34685725-kube-api-access-6jkgp\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.834444 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.834424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/03c047d0-92c4-43dc-8825-53ee34685725-signing-key\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.834970 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.834945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/03c047d0-92c4-43dc-8825-53ee34685725-signing-cabundle\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.836762 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.836737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/03c047d0-92c4-43dc-8825-53ee34685725-signing-key\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.847328 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.847309 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jkgp\" (UniqueName: \"kubernetes.io/projected/03c047d0-92c4-43dc-8825-53ee34685725-kube-api-access-6jkgp\") pod \"service-ca-79f68b9f4f-8mpmw\" (UID: \"03c047d0-92c4-43dc-8825-53ee34685725\") " pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:14.988931 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:14.988862 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" Jul 16 17:16:15.113934 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:15.113908 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-8mpmw"] Jul 16 17:16:15.116682 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:15.116654 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c047d0_92c4_43dc_8825_53ee34685725.slice/crio-f01b69dcf0f861cf6e2086dee50b063d0b1332aa18ea7dd95f070685e396b7f3 WatchSource:0}: Error finding container f01b69dcf0f861cf6e2086dee50b063d0b1332aa18ea7dd95f070685e396b7f3: Status 404 returned error can't find the container with id f01b69dcf0f861cf6e2086dee50b063d0b1332aa18ea7dd95f070685e396b7f3 Jul 16 17:16:15.944556 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:15.944517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:16:15.945027 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:15.944646 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 17:16:15.945027 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:15.944726 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls podName:eb84a249-ddf6-430e-bf26-2496fdf6b5bd nodeName:}" failed. No retries permitted until 2026-07-16 17:17:19.944705098 +0000 UTC m=+162.794211462 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls") pod "dns-default-4h2x5" (UID: "eb84a249-ddf6-430e-bf26-2496fdf6b5bd") : secret "dns-default-metrics-tls" not found Jul 16 17:16:16.065564 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:16.065528 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" event={"ID":"03c047d0-92c4-43dc-8825-53ee34685725","Type":"ContainerStarted","Data":"f01b69dcf0f861cf6e2086dee50b063d0b1332aa18ea7dd95f070685e396b7f3"} Jul 16 17:16:16.882416 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:16.882356 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:17.990414 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:17.990380 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6cn29" Jul 16 17:16:18.073955 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:18.073912 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" event={"ID":"03c047d0-92c4-43dc-8825-53ee34685725","Type":"ContainerStarted","Data":"2acbd7ea18563601bdecfb61ddb3bbf34be6eebe27d7c26ed1c7ef88dad457e2"} Jul 16 17:16:18.101802 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:18.101749 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-8mpmw" podStartSLOduration=2.018067068 podStartE2EDuration="4.101734743s" totalImagesPullingTime="2.083667675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:18.10014256 +0000 UTC m=+100.949648920" watchObservedRunningTime="2026-07-16 17:16:18.101734743 +0000 UTC m=+100.951241134" Jul 16 17:16:21.532005 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:21.531966 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:21.590432 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:21.590398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:21.590582 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:21.590497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:21.590622 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:21.590597 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 17:16:21.590660 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:21.590651 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls podName:f32c35b6-d851-4c34-b4b6-5586ca0e2357 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:37.590636649 +0000 UTC m=+120.440143015 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9qkd" (UID: "f32c35b6-d851-4c34-b4b6-5586ca0e2357") : secret "insights-runtime-extractor-tls" not found Jul 16 17:16:21.591046 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:21.591023 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f32c35b6-d851-4c34-b4b6-5586ca0e2357-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:24.108323 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:24.108296 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:24.207584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:24.207554 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:24.274232 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:24.274196 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:24.817431 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:24.817402 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:25.369408 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:25.369375 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:25.465388 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:25.465361 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:25.613216 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:25.613184 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:25.794074 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:25.794040 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:25.879002 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:25.878939 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:26.204810 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:26.204763 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:26.808493 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:26.808460 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:27.413539 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:27.413512 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:27.613752 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:27.613725 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:28.008273 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.008234 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:28.141095 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.141063 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:28.141247 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.141118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:28.141692 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.141662 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9790177d-d4dc-417f-aa42-f714a625d827-service-ca-bundle\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:28.143410 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.143389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9790177d-d4dc-417f-aa42-f714a625d827-metrics-certs\") pod \"router-default-568468f9f4-9h48n\" (UID: \"9790177d-d4dc-417f-aa42-f714a625d827\") " pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:28.218927 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.218895 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:28.398293 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.398266 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:28.526311 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.526278 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-568468f9f4-9h48n"] Jul 16 17:16:28.529010 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:28.528984 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9790177d_d4dc_417f_aa42_f714a625d827.slice/crio-1da1aa85c23bc3787312f041e230aeb21989f532801d4b7437ef9f9c51499b01 WatchSource:0}: Error finding container 1da1aa85c23bc3787312f041e230aeb21989f532801d4b7437ef9f9c51499b01: Status 404 returned error can't find the container with id 1da1aa85c23bc3787312f041e230aeb21989f532801d4b7437ef9f9c51499b01 Jul 16 17:16:28.615028 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:28.615001 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.081006 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.080968 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.107616 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.107580 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-568468f9f4-9h48n" event={"ID":"9790177d-d4dc-417f-aa42-f714a625d827","Type":"ContainerStarted","Data":"ab695fda444d6a59d7a4ce5b72795def4b5558dff067762e470fe9d2ef4b0954"} Jul 16 17:16:29.107743 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.107626 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-568468f9f4-9h48n" event={"ID":"9790177d-d4dc-417f-aa42-f714a625d827","Type":"ContainerStarted","Data":"1da1aa85c23bc3787312f041e230aeb21989f532801d4b7437ef9f9c51499b01"} Jul 16 17:16:29.138688 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.138644 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-568468f9f4-9h48n" podStartSLOduration=33.138629506 podStartE2EDuration="33.138629506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:15:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:29.138470714 +0000 UTC m=+111.987977075" watchObservedRunningTime="2026-07-16 17:16:29.138629506 +0000 UTC m=+111.988135845" Jul 16 17:16:29.170828 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.170783 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.270890 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.270863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.361679 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.361621 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.398764 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.398744 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:29.401153 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.401135 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:29.448760 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.448738 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.556051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.556025 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.653860 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.653834 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:29.749661 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:29.749632 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:30.110419 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:30.110339 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:30.111490 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:30.111470 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-568468f9f4-9h48n" Jul 16 17:16:30.351926 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:30.351895 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:30.454373 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:30.454346 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:30.908447 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:30.908417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:33.518391 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:33.518352 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:33.595261 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:33.595233 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:35.159673 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:35.159643 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:36.521844 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:36.521809 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:36.613402 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:36.613370 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:37.617730 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:37.617701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:37.620191 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:37.620153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f32c35b6-d851-4c34-b4b6-5586ca0e2357-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9qkd\" (UID: \"f32c35b6-d851-4c34-b4b6-5586ca0e2357\") " pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:37.635228 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:37.635206 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t9pcw\"" Jul 16 17:16:37.642015 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:37.641996 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q9qkd" Jul 16 17:16:37.738861 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:37.738829 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:37.778555 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:37.778514 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q9qkd"] Jul 16 17:16:37.783107 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:37.783077 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32c35b6_d851_4c34_b4b6_5586ca0e2357.slice/crio-f4312f8c1289b6ba599369f332e499b4ad7144e33ebf52d54dcac5950947cd44 WatchSource:0}: Error finding container f4312f8c1289b6ba599369f332e499b4ad7144e33ebf52d54dcac5950947cd44: Status 404 returned error can't find the container with id f4312f8c1289b6ba599369f332e499b4ad7144e33ebf52d54dcac5950947cd44 Jul 16 17:16:38.135256 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:38.135222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9qkd" event={"ID":"f32c35b6-d851-4c34-b4b6-5586ca0e2357","Type":"ContainerStarted","Data":"f4312f8c1289b6ba599369f332e499b4ad7144e33ebf52d54dcac5950947cd44"} Jul 16 17:16:38.855450 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:38.855417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:38.947886 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:38.947844 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:40.011552 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.011525 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-554c9df48d-7bwcr"] Jul 16 17:16:40.013583 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.013562 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.017119 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.017077 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 17:16:40.017229 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.017191 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqh7f\"" Jul 16 17:16:40.017229 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.017223 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 17:16:40.018473 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.018450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 17:16:40.024039 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.023999 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 17:16:40.035926 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.035898 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-554c9df48d-7bwcr"] Jul 16 17:16:40.098602 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.098571 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-554c9df48d-7bwcr"] Jul 16 17:16:40.098843 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:40.098822 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-j95kx registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" podUID="7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" Jul 16 17:16:40.138038 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138003 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-image-registry-private-configuration\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.138205 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-installation-pull-secrets\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.138205 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-tls\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.138205 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j95kx\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-kube-api-access-j95kx\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.138370 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-certificates\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.138370 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138281 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-trusted-ca\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.138370 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138304 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-ca-trust-extracted\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.138370 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.138327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-bound-sa-token\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.141518 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.141491 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.147297 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.147276 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239136 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239060 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-trusted-ca\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239136 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-ca-trust-extracted\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239362 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239151 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-bound-sa-token\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239362 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-image-registry-private-configuration\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239457 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-installation-pull-secrets\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239508 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239468 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-tls\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239592 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j95kx\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-kube-api-access-j95kx\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.239657 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.239614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-certificates\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.240315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.240254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-ca-trust-extracted\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.240315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.240267 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-trusted-ca\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.241002 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.240945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-certificates\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.241686 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.241657 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:40.242196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.242154 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-image-registry-private-configuration\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.242361 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.242325 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-tls\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.242692 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.242668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-installation-pull-secrets\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.261193 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.261167 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-bound-sa-token\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.261325 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.261312 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j95kx\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-kube-api-access-j95kx\") pod \"image-registry-554c9df48d-7bwcr\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:40.337306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.337270 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:40.340114 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340084 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-ca-trust-extracted\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340130 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-bound-sa-token\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340168 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-trusted-ca\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340202 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-image-registry-private-configuration\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340230 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-installation-pull-secrets\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340265 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-certificates\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340561 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340304 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-tls\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340561 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j95kx\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-kube-api-access-j95kx\") pod \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\" (UID: \"7d745fc7-c560-4cd2-8c4e-cb02bd652e1e\") " Jul 16 17:16:40.340561 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340371 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:16:40.340561 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340538 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-ca-trust-extracted\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:40.340757 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340676 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:16:40.340757 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.340721 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:16:40.343294 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.343244 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:16:40.343294 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.343284 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:16:40.343469 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.343336 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:16:40.343469 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.343402 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:16:40.343559 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.343511 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-kube-api-access-j95kx" (OuterVolumeSpecName: "kube-api-access-j95kx") pod "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" (UID: "7d745fc7-c560-4cd2-8c4e-cb02bd652e1e"). InnerVolumeSpecName "kube-api-access-j95kx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:16:40.441604 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.441568 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-trusted-ca\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:40.441604 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.441605 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-image-registry-private-configuration\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:40.441604 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.441621 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-installation-pull-secrets\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:40.441921 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.441635 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-certificates\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:40.441921 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.441649 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-registry-tls\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:40.441921 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.441663 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j95kx\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-kube-api-access-j95kx\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:40.441921 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:40.441676 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e-bound-sa-token\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:16:41.145247 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:41.145218 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554c9df48d-7bwcr" Jul 16 17:16:41.203548 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:41.203510 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-554c9df48d-7bwcr"] Jul 16 17:16:41.208735 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:41.208707 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-554c9df48d-7bwcr"] Jul 16 17:16:41.685246 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:41.685207 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d745fc7-c560-4cd2-8c4e-cb02bd652e1e" path="/var/lib/kubelet/pods/7d745fc7-c560-4cd2-8c4e-cb02bd652e1e/volumes" Jul 16 17:16:43.152572 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:43.152541 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9qkd" event={"ID":"f32c35b6-d851-4c34-b4b6-5586ca0e2357","Type":"ContainerStarted","Data":"1fd3513015c103c43161825f7de38d9704bd2d50a60316471f7f32d6d1baaaf7"} Jul 16 17:16:43.155748 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:43.155725 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:43.260406 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:43.260372 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:46.219985 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.219961 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:46.326233 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.326200 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-z55bt"] Jul 16 17:16:46.328023 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.328002 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.328701 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.328678 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:46.333088 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.333071 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 16 17:16:46.334292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.334241 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 17:16:46.334292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.334271 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 16 17:16:46.335249 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.335228 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 17:16:46.335249 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.335242 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 17:16:46.335398 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.335230 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p42rx\"" Jul 16 17:16:46.354944 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.354923 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-z55bt"] Jul 16 17:16:46.488280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.488185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.488280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.488234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:16:46.488280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.488272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-metrics-client-ca\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.488509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.488303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.488509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.488340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwwr9\" (UniqueName: \"kubernetes.io/projected/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-kube-api-access-bwwr9\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.490458 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.490428 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26b0b5ae-c523-4603-b958-bf5e8b6017cc-metrics-certs\") pod \"network-metrics-daemon-27zcc\" (UID: \"26b0b5ae-c523-4603-b958-bf5e8b6017cc\") " pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:16:46.562099 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.562062 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:46.589087 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.589025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.589087 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.589081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-metrics-client-ca\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.589292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.589116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.589292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.589158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwwr9\" (UniqueName: \"kubernetes.io/projected/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-kube-api-access-bwwr9\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.590019 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.589995 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-metrics-client-ca\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.591647 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.591627 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.591767 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.591744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.609836 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.609808 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwwr9\" (UniqueName: \"kubernetes.io/projected/5801e7a4-b71f-4259-92e0-6f9fe9f159cb-kube-api-access-bwwr9\") pod \"prometheus-operator-64785b6675-z55bt\" (UID: \"5801e7a4-b71f-4259-92e0-6f9fe9f159cb\") " pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.610487 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.610462 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-65bh8\"" Jul 16 17:16:46.617339 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.617315 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-27zcc" Jul 16 17:16:46.638722 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.638693 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" Jul 16 17:16:46.774743 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.774664 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-27zcc"] Jul 16 17:16:46.780676 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:46.780641 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b0b5ae_c523_4603_b958_bf5e8b6017cc.slice/crio-ebeef0ec6f9d7a04241c2cf1f8bc4fd5fae767141fb9b525c1be2e1701e60729 WatchSource:0}: Error finding container ebeef0ec6f9d7a04241c2cf1f8bc4fd5fae767141fb9b525c1be2e1701e60729: Status 404 returned error can't find the container with id ebeef0ec6f9d7a04241c2cf1f8bc4fd5fae767141fb9b525c1be2e1701e60729 Jul 16 17:16:46.804353 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:46.804315 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-z55bt"] Jul 16 17:16:46.807487 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:46.807459 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5801e7a4_b71f_4259_92e0_6f9fe9f159cb.slice/crio-3a6bcf62659f2c15f9a928e71ffa41e80031f5a2db25ec47b2579bf82d3589bb WatchSource:0}: Error finding container 3a6bcf62659f2c15f9a928e71ffa41e80031f5a2db25ec47b2579bf82d3589bb: Status 404 returned error can't find the container with id 3a6bcf62659f2c15f9a928e71ffa41e80031f5a2db25ec47b2579bf82d3589bb Jul 16 17:16:47.169840 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.169770 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9qkd" event={"ID":"f32c35b6-d851-4c34-b4b6-5586ca0e2357","Type":"ContainerStarted","Data":"ca9e402ce1dff0c1e3fe8d1bef189d611db7abe72170f309adf2cc819eeb370e"} Jul 16 17:16:47.171178 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.171156 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" event={"ID":"5801e7a4-b71f-4259-92e0-6f9fe9f159cb","Type":"ContainerStarted","Data":"3a6bcf62659f2c15f9a928e71ffa41e80031f5a2db25ec47b2579bf82d3589bb"} Jul 16 17:16:47.172826 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.172775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-27zcc" event={"ID":"26b0b5ae-c523-4603-b958-bf5e8b6017cc","Type":"ContainerStarted","Data":"ebeef0ec6f9d7a04241c2cf1f8bc4fd5fae767141fb9b525c1be2e1701e60729"} Jul 16 17:16:47.767682 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.767277 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-757b895bf9-d7mk4"] Jul 16 17:16:47.770116 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.770096 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:47.773081 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.773060 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 17:16:47.774256 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.774235 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 17:16:47.774591 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.774569 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 17:16:47.778918 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.778893 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 17:16:47.779267 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.779105 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 17:16:47.779851 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.779527 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 17:16:47.779851 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.779741 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 17:16:47.782403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.782385 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z2nks\"" Jul 16 17:16:47.792213 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.789929 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-757b895bf9-d7mk4"] Jul 16 17:16:47.792213 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.790990 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 17:16:47.899985 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.899941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-service-ca\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:47.900171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.900033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-console-config\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:47.900171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.900055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-trusted-ca-bundle\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:47.900171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.900087 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-oauth-config\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:47.900171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.900103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-serving-cert\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:47.900171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.900119 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-oauth-serving-cert\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:47.900171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:47.900139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc9m2\" (UniqueName: \"kubernetes.io/projected/c63f860e-efee-41d4-8767-5169b323692d-kube-api-access-pc9m2\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.001520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.001109 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc9m2\" (UniqueName: \"kubernetes.io/projected/c63f860e-efee-41d4-8767-5169b323692d-kube-api-access-pc9m2\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.001520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.001165 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-service-ca\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.001520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.001231 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-console-config\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.001520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.001252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-trusted-ca-bundle\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.001520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.001301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-oauth-config\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.001520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.001326 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-serving-cert\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.001520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.001348 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-oauth-serving-cert\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.002638 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.002008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-service-ca\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.002638 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.002596 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-console-config\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.002845 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.002687 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-trusted-ca-bundle\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.003242 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.003209 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-oauth-serving-cert\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.004374 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.004348 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-oauth-config\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.004562 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.004536 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-serving-cert\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.016623 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.016590 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc9m2\" (UniqueName: \"kubernetes.io/projected/c63f860e-efee-41d4-8767-5169b323692d-kube-api-access-pc9m2\") pod \"console-757b895bf9-d7mk4\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.094048 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.094008 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:48.178148 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.178113 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-27zcc" event={"ID":"26b0b5ae-c523-4603-b958-bf5e8b6017cc","Type":"ContainerStarted","Data":"ed22677a2e68435c251caf9d9d33b896692c64495abe39c9fd8ac52bb36d6d36"} Jul 16 17:16:48.304912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:48.304747 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-757b895bf9-d7mk4"] Jul 16 17:16:49.446738 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:49.446687 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:49.553117 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:49.553050 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:49.822861 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:49.822770 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63f860e_efee_41d4_8767_5169b323692d.slice/crio-b5018cc253c6c8bf173db419045410be9cfc0ef6763ffa3312913cdfb0aed908 WatchSource:0}: Error finding container b5018cc253c6c8bf173db419045410be9cfc0ef6763ffa3312913cdfb0aed908: Status 404 returned error can't find the container with id b5018cc253c6c8bf173db419045410be9cfc0ef6763ffa3312913cdfb0aed908 Jul 16 17:16:49.898475 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:49.898444 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:50.011987 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.011951 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:50.187285 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.187245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9qkd" event={"ID":"f32c35b6-d851-4c34-b4b6-5586ca0e2357","Type":"ContainerStarted","Data":"6b1ac0bf3c7c5eaadaafd968dd072ca69ef821247fff7bd2e5e3230dc923998d"} Jul 16 17:16:50.188528 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.188498 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-757b895bf9-d7mk4" event={"ID":"c63f860e-efee-41d4-8767-5169b323692d","Type":"ContainerStarted","Data":"b5018cc253c6c8bf173db419045410be9cfc0ef6763ffa3312913cdfb0aed908"} Jul 16 17:16:50.190173 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.190142 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" event={"ID":"5801e7a4-b71f-4259-92e0-6f9fe9f159cb","Type":"ContainerStarted","Data":"5e45a5930b41461b7ed3d0d8ebe2e6695aee9bf8d36e7842cb10b3da27ab26ed"} Jul 16 17:16:50.190280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.190202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" event={"ID":"5801e7a4-b71f-4259-92e0-6f9fe9f159cb","Type":"ContainerStarted","Data":"a530a5bb4e295d04ea864decc8e4a1a21315ecd2bb97ea73fc76cb80bc92f5dc"} Jul 16 17:16:50.191949 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.191927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-27zcc" event={"ID":"26b0b5ae-c523-4603-b958-bf5e8b6017cc","Type":"ContainerStarted","Data":"3175741d03df1a14fb98e6ba0f1964b803d073d1c80d756d3632228d0708b6b5"} Jul 16 17:16:50.211048 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.211020 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:50.220920 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.220880 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q9qkd" podStartSLOduration=33.256795995 podStartE2EDuration="45.220867622s" totalImagesPullingTime="11.964071627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 17:16:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:50.220828464 +0000 UTC m=+133.070334825" watchObservedRunningTime="2026-07-16 17:16:50.220867622 +0000 UTC m=+133.070373983" Jul 16 17:16:50.251024 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.250971 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-27zcc" podStartSLOduration=131.327078594 podStartE2EDuration="2m12.250954579s" totalImagesPullingTime="923.875985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:50.250923235 +0000 UTC m=+133.100429596" watchObservedRunningTime="2026-07-16 17:16:50.250954579 +0000 UTC m=+133.100460939" Jul 16 17:16:50.285169 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.285113 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-z55bt" podStartSLOduration=1.270564608 podStartE2EDuration="4.285097898s" totalImagesPullingTime="3.01453329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:50.283779728 +0000 UTC m=+133.133286124" watchObservedRunningTime="2026-07-16 17:16:50.285097898 +0000 UTC m=+133.134604259" Jul 16 17:16:50.461241 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.461151 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:50.559188 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:50.559152 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:51.376885 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:51.376850 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:51.463484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:51.463444 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:52.490494 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:52.490468 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:52.867533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:52.867454 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:52.973030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:52.972998 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:53.044594 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.044563 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-chbml"] Jul 16 17:16:53.047283 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.047265 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.050535 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.050517 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 17:16:53.050748 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.050730 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 17:16:53.054568 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.054547 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b285m\"" Jul 16 17:16:53.054650 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.054609 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 17:16:53.061500 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.061480 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-8cvcc"] Jul 16 17:16:53.064509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.064494 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.067519 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.067497 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 16 17:16:53.068146 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.068128 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7qr79\"" Jul 16 17:16:53.068290 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.068276 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 16 17:16:53.075387 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.075371 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 16 17:16:53.101424 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.101343 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:53.102284 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.102264 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-8cvcc"] Jul 16 17:16:53.146903 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.146876 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147038 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.146919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-accelerators-collector-config\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147038 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.146968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dsx9\" (UniqueName: \"kubernetes.io/projected/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-api-access-7dsx9\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.147038 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-sys\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147158 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.147158 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-textfile\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147158 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.147158 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.147158 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147133 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f67e6150-2be9-40f6-825a-25944c25450d-metrics-client-ca\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147158 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147151 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-wtmp\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147396 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-root\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147396 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147183 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.147396 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.147396 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147274 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-tls\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.147396 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.147330 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-strgm\" (UniqueName: \"kubernetes.io/projected/f67e6150-2be9-40f6-825a-25944c25450d-kube-api-access-strgm\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.193376 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.193336 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:53.205784 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.205757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-757b895bf9-d7mk4" event={"ID":"c63f860e-efee-41d4-8767-5169b323692d","Type":"ContainerStarted","Data":"0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a"} Jul 16 17:16:53.247902 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.247866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-textfile\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.247911 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.248063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.247981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.248063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f67e6150-2be9-40f6-825a-25944c25450d-metrics-client-ca\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248035 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-wtmp\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248062 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-root\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248088 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.248306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.248306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248183 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-tls\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-strgm\" (UniqueName: \"kubernetes.io/projected/f67e6150-2be9-40f6-825a-25944c25450d-kube-api-access-strgm\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248249 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-accelerators-collector-config\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248568 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dsx9\" (UniqueName: \"kubernetes.io/projected/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-api-access-7dsx9\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.248568 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-sys\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.248568 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.248387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.249161 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.249138 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.249406 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.249386 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-textfile\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.249930 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.249902 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-wtmp\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.250426 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.250391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-root\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.250928 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.250904 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.251391 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.251368 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f67e6150-2be9-40f6-825a-25944c25450d-metrics-client-ca\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.252633 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.251812 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-accelerators-collector-config\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.252633 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.252412 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f67e6150-2be9-40f6-825a-25944c25450d-sys\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.256828 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.253151 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.256828 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.254955 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-tls\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.264332 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.264302 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.264731 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.264707 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.267014 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.266988 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f67e6150-2be9-40f6-825a-25944c25450d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.268593 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.268571 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-strgm\" (UniqueName: \"kubernetes.io/projected/f67e6150-2be9-40f6-825a-25944c25450d-kube-api-access-strgm\") pod \"node-exporter-chbml\" (UID: \"f67e6150-2be9-40f6-825a-25944c25450d\") " pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.278152 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.278133 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dsx9\" (UniqueName: \"kubernetes.io/projected/11dad2a0-1fb6-4fd7-a197-3ba2757fc659-kube-api-access-7dsx9\") pod \"kube-state-metrics-785778ff74-8cvcc\" (UID: \"11dad2a0-1fb6-4fd7-a197-3ba2757fc659\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.356855 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.356829 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-chbml" Jul 16 17:16:53.367204 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:53.367179 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf67e6150_2be9_40f6_825a_25944c25450d.slice/crio-6c8be6231ea6ca93eb89e94ce32b66e30b4108a093faaaff9cde551393ca762d WatchSource:0}: Error finding container 6c8be6231ea6ca93eb89e94ce32b66e30b4108a093faaaff9cde551393ca762d: Status 404 returned error can't find the container with id 6c8be6231ea6ca93eb89e94ce32b66e30b4108a093faaaff9cde551393ca762d Jul 16 17:16:53.373162 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.373142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" Jul 16 17:16:53.500656 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.500624 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:53.537434 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.537383 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-757b895bf9-d7mk4" podStartSLOduration=3.942355557 podStartE2EDuration="6.53736226s" totalImagesPullingTime="2.595006703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:53.242041958 +0000 UTC m=+136.091548320" watchObservedRunningTime="2026-07-16 17:16:53.53736226 +0000 UTC m=+136.386868622" Jul 16 17:16:53.538028 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.538009 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-8cvcc"] Jul 16 17:16:53.540644 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:53.540615 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11dad2a0_1fb6_4fd7_a197_3ba2757fc659.slice/crio-a547056b5cc60a88a86255d274fa07a924709f012ae305889b2e05fdd797de49 WatchSource:0}: Error finding container a547056b5cc60a88a86255d274fa07a924709f012ae305889b2e05fdd797de49: Status 404 returned error can't find the container with id a547056b5cc60a88a86255d274fa07a924709f012ae305889b2e05fdd797de49 Jul 16 17:16:53.867882 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.867774 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:53.972041 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:53.972006 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:54.056718 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.056679 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:16:54.060385 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.060362 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.065344 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.064350 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 17:16:54.065344 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.064854 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 17:16:54.065344 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.065193 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 17:16:54.065599 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.065429 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 17:16:54.065599 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.065454 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 17:16:54.065599 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.065506 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 17:16:54.065748 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.065699 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 17:16:54.065822 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.065768 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pb42v\"" Jul 16 17:16:54.065932 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.065915 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 17:16:54.066046 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.066024 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 17:16:54.107330 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.107294 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:16:54.110510 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.110481 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:54.155577 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.155740 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqnll\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-kube-api-access-lqnll\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.155740 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.155740 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155722 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.155899 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155780 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.155899 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155849 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.155899 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155877 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.156032 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.156032 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-config-out\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.156032 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.155994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.156151 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.156042 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.156151 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.156074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-web-config\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.156239 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.156179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.211681 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.211640 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-chbml" event={"ID":"f67e6150-2be9-40f6-825a-25944c25450d","Type":"ContainerStarted","Data":"6c8be6231ea6ca93eb89e94ce32b66e30b4108a093faaaff9cde551393ca762d"} Jul 16 17:16:54.213048 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.213015 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" event={"ID":"11dad2a0-1fb6-4fd7-a197-3ba2757fc659","Type":"ContainerStarted","Data":"a547056b5cc60a88a86255d274fa07a924709f012ae305889b2e05fdd797de49"} Jul 16 17:16:54.257558 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.257742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.257742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-web-config\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.257742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.257742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257718 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.257742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqnll\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-kube-api-access-lqnll\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.257979 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.258011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.258037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258370 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.258065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-config-out\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.258934 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.258904 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.259223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.259195 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.261022 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.260986 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-config-out\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.261694 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.261594 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.261694 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.261650 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.262107 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.262013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.262424 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.262375 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.262768 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.262695 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.263170 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.263120 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.263170 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.263145 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-web-config\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.263491 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.263474 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.285335 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.285306 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqnll\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-kube-api-access-lqnll\") pod \"alertmanager-main-0\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.373593 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.373569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:16:54.574519 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.574482 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:16:54.577549 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:54.577526 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd421f33f_6a4f_4086_8268_d2e802be27e7.slice/crio-7af71f0d6b90dfd8ab6a251b383cbfdf328c75c673b31bfc5afe19368c0ea9b0 WatchSource:0}: Error finding container 7af71f0d6b90dfd8ab6a251b383cbfdf328c75c673b31bfc5afe19368c0ea9b0: Status 404 returned error can't find the container with id 7af71f0d6b90dfd8ab6a251b383cbfdf328c75c673b31bfc5afe19368c0ea9b0 Jul 16 17:16:54.776286 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.776201 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:54.926579 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:54.926550 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:55.218679 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:55.218649 2577 generic.go:356] "Generic (PLEG): container finished" podID="f67e6150-2be9-40f6-825a-25944c25450d" containerID="9a2e84e625a41b7e25fc0c85f675239a88b4524ebe999861de6bbc3183ef902e" exitCode=0 Jul 16 17:16:55.218878 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:55.218730 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-chbml" event={"ID":"f67e6150-2be9-40f6-825a-25944c25450d","Type":"ContainerDied","Data":"9a2e84e625a41b7e25fc0c85f675239a88b4524ebe999861de6bbc3183ef902e"} Jul 16 17:16:55.220457 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:55.220421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerStarted","Data":"7af71f0d6b90dfd8ab6a251b383cbfdf328c75c673b31bfc5afe19368c0ea9b0"} Jul 16 17:16:55.705090 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:55.705053 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:55.842934 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:55.842849 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:55.989262 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:55.989230 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:56.169270 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:56.169237 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:56.225759 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:56.225725 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-chbml" event={"ID":"f67e6150-2be9-40f6-825a-25944c25450d","Type":"ContainerStarted","Data":"3a849c4602e44005fb04882dfc6c472419647cc42947ee42de56f473097e50e7"} Jul 16 17:16:56.225946 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:56.225807 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-chbml" event={"ID":"f67e6150-2be9-40f6-825a-25944c25450d","Type":"ContainerStarted","Data":"b0829a1f9b6b035c92f49aa944425e0f4c1804e33b2d84889dd64dcb860de6e7"} Jul 16 17:16:56.227304 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:56.227278 2577 generic.go:356] "Generic (PLEG): container finished" podID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerID="d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de" exitCode=0 Jul 16 17:16:56.227404 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:56.227316 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de"} Jul 16 17:16:56.256330 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:56.256173 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-chbml" podStartSLOduration=2.217024051 podStartE2EDuration="3.256155597s" totalImagesPullingTime="965.139503ms" totalInitContainerRuntime="73.992043ms" isStatefulPod=true podCreationTimestamp="2026-07-16 17:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:56.254743117 +0000 UTC m=+139.104249478" watchObservedRunningTime="2026-07-16 17:16:56.256155597 +0000 UTC m=+139.105661959" Jul 16 17:16:56.262536 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:56.262512 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:57.234762 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.234729 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:57.637973 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.637942 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-757b895bf9-d7mk4"] Jul 16 17:16:57.659825 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.659773 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld"] Jul 16 17:16:57.664013 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.663978 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:57.666838 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.666803 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 16 17:16:57.667175 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.667157 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-b5qxc\"" Jul 16 17:16:57.687520 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.687492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zpmld\" (UID: \"7d8b5e0c-5719-4f22-98fe-7a64f3e53277\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:57.687668 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.687532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdl2d\" (UniqueName: \"kubernetes.io/projected/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-kube-api-access-cdl2d\") pod \"monitoring-plugin-5f68df7f58-zpmld\" (UID: \"7d8b5e0c-5719-4f22-98fe-7a64f3e53277\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:57.690505 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.690480 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld"] Jul 16 17:16:57.788543 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.788508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zpmld\" (UID: \"7d8b5e0c-5719-4f22-98fe-7a64f3e53277\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:57.788647 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.788555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdl2d\" (UniqueName: \"kubernetes.io/projected/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-kube-api-access-cdl2d\") pod \"monitoring-plugin-5f68df7f58-zpmld\" (UID: \"7d8b5e0c-5719-4f22-98fe-7a64f3e53277\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:57.788719 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:57.788696 2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 16 17:16:57.789448 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:16:57.788983 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-monitoring-plugin-cert podName:7d8b5e0c-5719-4f22-98fe-7a64f3e53277 nodeName:}" failed. No retries permitted until 2026-07-16 17:16:58.288917087 +0000 UTC m=+141.138423433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-zpmld" (UID: "7d8b5e0c-5719-4f22-98fe-7a64f3e53277") : secret "monitoring-plugin-cert" not found Jul 16 17:16:57.843494 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:57.843418 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdl2d\" (UniqueName: \"kubernetes.io/projected/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-kube-api-access-cdl2d\") pod \"monitoring-plugin-5f68df7f58-zpmld\" (UID: \"7d8b5e0c-5719-4f22-98fe-7a64f3e53277\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:58.094761 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.094724 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:16:58.239259 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.239159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerStarted","Data":"3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290"} Jul 16 17:16:58.239259 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.239242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerStarted","Data":"9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570"} Jul 16 17:16:58.239730 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.239283 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerStarted","Data":"d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f"} Jul 16 17:16:58.239730 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.239318 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerStarted","Data":"b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a"} Jul 16 17:16:58.239730 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.239358 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerStarted","Data":"656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970"} Jul 16 17:16:58.241493 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.241463 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" event={"ID":"11dad2a0-1fb6-4fd7-a197-3ba2757fc659","Type":"ContainerStarted","Data":"db2e76a0496ae199aa7bb8b7f277a9f7687554c18360fb18be7b63ee2c399523"} Jul 16 17:16:58.241595 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.241512 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" event={"ID":"11dad2a0-1fb6-4fd7-a197-3ba2757fc659","Type":"ContainerStarted","Data":"ae9263bdd56d5adeb3cc8fafb9ecf9df37080974ae48d43c84f25f77d14fb5a6"} Jul 16 17:16:58.241595 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.241533 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" event={"ID":"11dad2a0-1fb6-4fd7-a197-3ba2757fc659","Type":"ContainerStarted","Data":"b423cf3b5c370ec70a61b3e1d7070e8017fbc0da5b6c79f9a63712bbd137ff1f"} Jul 16 17:16:58.264426 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.264379 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-8cvcc" podStartSLOduration=1.167418146 podStartE2EDuration="5.264359439s" totalImagesPullingTime="4.096941293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:58.263687662 +0000 UTC m=+141.113194023" watchObservedRunningTime="2026-07-16 17:16:58.264359439 +0000 UTC m=+141.113865807" Jul 16 17:16:58.293777 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.293733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zpmld\" (UID: \"7d8b5e0c-5719-4f22-98fe-7a64f3e53277\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:58.296216 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.296186 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d8b5e0c-5719-4f22-98fe-7a64f3e53277-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-zpmld\" (UID: \"7d8b5e0c-5719-4f22-98fe-7a64f3e53277\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:58.326899 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.326866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:16:58.458004 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:58.457971 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld"] Jul 16 17:16:58.461048 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:58.461022 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8b5e0c_5719_4f22_98fe_7a64f3e53277.slice/crio-d9d75d16b6b2b18bc56fc3f2a96d1eaca869ee9d726f20316ebf4f98c2ad2ee5 WatchSource:0}: Error finding container d9d75d16b6b2b18bc56fc3f2a96d1eaca869ee9d726f20316ebf4f98c2ad2ee5: Status 404 returned error can't find the container with id d9d75d16b6b2b18bc56fc3f2a96d1eaca869ee9d726f20316ebf4f98c2ad2ee5 Jul 16 17:16:59.250717 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.250678 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerStarted","Data":"6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2"} Jul 16 17:16:59.252954 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.252804 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" event={"ID":"7d8b5e0c-5719-4f22-98fe-7a64f3e53277","Type":"ContainerStarted","Data":"d9d75d16b6b2b18bc56fc3f2a96d1eaca869ee9d726f20316ebf4f98c2ad2ee5"} Jul 16 17:16:59.296651 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.296602 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.062309798 podStartE2EDuration="5.296584956s" totalImagesPullingTime="3.219050592s" totalInitContainerRuntime="15.224566ms" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:16:59.292853879 +0000 UTC m=+142.142360240" watchObservedRunningTime="2026-07-16 17:16:59.296584956 +0000 UTC m=+142.146091317" Jul 16 17:16:59.428910 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.428870 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:16:59.434101 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.434067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.438945 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.438917 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 17:16:59.439140 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.438971 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 17:16:59.439259 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.439195 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 17:16:59.439259 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.439242 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 17:16:59.439394 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.439019 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 17:16:59.439394 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.439045 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 17:16:59.439498 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.439079 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 17:16:59.439498 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.439469 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 17:16:59.439498 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.439151 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 17:16:59.439636 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.438982 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 17:16:59.440336 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.440317 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 17:16:59.441107 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.440570 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 17:16:59.441107 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.440703 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 17:16:59.441107 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.440734 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6e9mgf33es629\"" Jul 16 17:16:59.441107 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.440966 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vk8jr\"" Jul 16 17:16:59.443572 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.443552 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 17:16:59.461712 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.461683 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 17:16:59.462455 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.462425 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:16:59.504836 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.504720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.504856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.504890 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.504910 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28qz8\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-kube-api-access-28qz8\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.504941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.504977 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505036 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505105 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505208 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505231 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505318 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505293 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505723 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505723 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505723 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505433 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.505723 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.505465 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.606315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.606282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.606315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.606318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.606525 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.606338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.606525 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.606374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.606525 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.606401 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.606525 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.606430 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607017 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.606991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607123 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607123 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607123 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607272 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607272 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607185 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607272 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607272 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607229 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607272 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607299 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607386 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607463 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28qz8\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-kube-api-access-28qz8\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.607509 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.607490 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.608106 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.608085 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.609049 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.609025 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.609481 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.609461 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.609807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.609768 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.610484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.610153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-web-config\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.610484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.610188 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config-out\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.610484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.610187 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.610703 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.610654 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.612907 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.612822 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.613003 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.612962 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.613110 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.613088 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.613336 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.613318 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.615030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.615008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.615247 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.615225 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.615415 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.615395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.615854 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.615835 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.623909 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.623871 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28qz8\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-kube-api-access-28qz8\") pod \"prometheus-k8s-0\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.667283 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.667249 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:59.747892 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.747856 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:16:59.749556 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.749527 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:16:59.918927 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:16:59.918905 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:16:59.920211 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:16:59.920180 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ab787f_9002_4b69_8236_d80a65ef1dfc.slice/crio-0269006cf0348ac2249bdedb5281c21cb5def9a7f9cd5d6c2669d5d41bab2b3e WatchSource:0}: Error finding container 0269006cf0348ac2249bdedb5281c21cb5def9a7f9cd5d6c2669d5d41bab2b3e: Status 404 returned error can't find the container with id 0269006cf0348ac2249bdedb5281c21cb5def9a7f9cd5d6c2669d5d41bab2b3e Jul 16 17:17:00.203981 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:00.203950 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:00.257925 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:00.257894 2577 generic.go:356] "Generic (PLEG): container finished" podID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" exitCode=0 Jul 16 17:17:00.258263 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:00.257984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882"} Jul 16 17:17:00.258263 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:00.258058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerStarted","Data":"0269006cf0348ac2249bdedb5281c21cb5def9a7f9cd5d6c2669d5d41bab2b3e"} Jul 16 17:17:00.302082 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:00.302053 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:02.268977 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:02.268935 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" event={"ID":"7d8b5e0c-5719-4f22-98fe-7a64f3e53277","Type":"ContainerStarted","Data":"0f7c57222f5bd0b201e25fe404812ffa45a3f73a547c1bc6b5f4a6453cf8d16e"} Jul 16 17:17:02.269400 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:02.269134 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:17:02.275591 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:02.275569 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" Jul 16 17:17:02.312054 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:02.311969 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-zpmld" podStartSLOduration=1.59703951 podStartE2EDuration="5.311951412s" totalImagesPullingTime="3.714911902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:02.310111477 +0000 UTC m=+145.159617841" watchObservedRunningTime="2026-07-16 17:17:02.311951412 +0000 UTC m=+145.161457774" Jul 16 17:17:04.279507 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:04.279463 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerStarted","Data":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} Jul 16 17:17:04.279985 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:04.279561 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerStarted","Data":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} Jul 16 17:17:06.017246 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:06.017192 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:06.291341 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:06.291265 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerStarted","Data":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} Jul 16 17:17:06.291476 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:06.291344 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerStarted","Data":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} Jul 16 17:17:06.291476 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:06.291384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerStarted","Data":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} Jul 16 17:17:06.291476 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:06.291424 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerStarted","Data":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} Jul 16 17:17:06.356108 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:06.356042 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.17846498 podStartE2EDuration="7.356019812s" totalImagesPullingTime="5.157366943s" totalInitContainerRuntime="20.187889ms" isStatefulPod=false podCreationTimestamp="2026-07-16 17:16:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:06.352760162 +0000 UTC m=+149.202266522" watchObservedRunningTime="2026-07-16 17:17:06.356019812 +0000 UTC m=+149.205526177" Jul 16 17:17:08.122465 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:08.122428 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:08.203562 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:08.203517 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:09.170086 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:09.170049 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:09.245378 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:09.245342 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:09.748484 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:09.748444 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:17:14.112367 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:14.112331 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:14.233465 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:14.233435 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:14.307244 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:14.307211 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:15.083060 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:17:15.083020 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4h2x5" podUID="eb84a249-ddf6-430e-bf26-2496fdf6b5bd" Jul 16 17:17:15.324006 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:15.323977 2577 generic.go:356] "Generic (PLEG): container finished" podID="53f52dc0-37b5-4f63-814c-599b8fd74feb" containerID="290c60649acfe70764533ab4a109aee98bcab136b4e893cf34a7930fb192a2b9" exitCode=0 Jul 16 17:17:15.324334 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:15.324051 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" event={"ID":"53f52dc0-37b5-4f63-814c-599b8fd74feb","Type":"ContainerDied","Data":"290c60649acfe70764533ab4a109aee98bcab136b4e893cf34a7930fb192a2b9"} Jul 16 17:17:15.324334 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:15.324088 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4h2x5" Jul 16 17:17:15.324511 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:15.324495 2577 scope.go:117] "RemoveContainer" containerID="290c60649acfe70764533ab4a109aee98bcab136b4e893cf34a7930fb192a2b9" Jul 16 17:17:16.329670 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:16.329633 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-dwrfw" event={"ID":"53f52dc0-37b5-4f63-814c-599b8fd74feb","Type":"ContainerStarted","Data":"31646d14e3d00ecd073840c189ad2f52d80d120ff5a40f8881496f6c882574be"} Jul 16 17:17:19.998849 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:19.998809 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:17:20.001012 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:20.000990 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb84a249-ddf6-430e-bf26-2496fdf6b5bd-metrics-tls\") pod \"dns-default-4h2x5\" (UID: \"eb84a249-ddf6-430e-bf26-2496fdf6b5bd\") " pod="openshift-dns/dns-default-4h2x5" Jul 16 17:17:20.131044 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:20.131012 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bfmt5\"" Jul 16 17:17:20.135914 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:20.135895 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4h2x5" Jul 16 17:17:20.449415 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:20.449376 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4h2x5"] Jul 16 17:17:20.452419 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:17:20.452394 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb84a249_ddf6_430e_bf26_2496fdf6b5bd.slice/crio-a683c5a25f780b53e11c39545ba23cf5a9766fa81bbbd482e2af9a544008c5f5 WatchSource:0}: Error finding container a683c5a25f780b53e11c39545ba23cf5a9766fa81bbbd482e2af9a544008c5f5: Status 404 returned error can't find the container with id a683c5a25f780b53e11c39545ba23cf5a9766fa81bbbd482e2af9a544008c5f5 Jul 16 17:17:21.351941 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:21.351895 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4h2x5" event={"ID":"eb84a249-ddf6-430e-bf26-2496fdf6b5bd","Type":"ContainerStarted","Data":"a683c5a25f780b53e11c39545ba23cf5a9766fa81bbbd482e2af9a544008c5f5"} Jul 16 17:17:21.539952 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:21.539919 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:21.618464 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:21.618377 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:22.661323 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:22.661283 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-757b895bf9-d7mk4" podUID="c63f860e-efee-41d4-8767-5169b323692d" containerName="console" containerID="cri-o://0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a" gracePeriod=15 Jul 16 17:17:22.966887 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:22.966862 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:17:23.127535 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.127501 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-trusted-ca-bundle\") pod \"c63f860e-efee-41d4-8767-5169b323692d\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " Jul 16 17:17:23.127664 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.127565 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-serving-cert\") pod \"c63f860e-efee-41d4-8767-5169b323692d\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " Jul 16 17:17:23.127664 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.127588 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-console-config\") pod \"c63f860e-efee-41d4-8767-5169b323692d\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " Jul 16 17:17:23.127664 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.127635 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-service-ca\") pod \"c63f860e-efee-41d4-8767-5169b323692d\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " Jul 16 17:17:23.127827 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.127674 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc9m2\" (UniqueName: \"kubernetes.io/projected/c63f860e-efee-41d4-8767-5169b323692d-kube-api-access-pc9m2\") pod \"c63f860e-efee-41d4-8767-5169b323692d\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " Jul 16 17:17:23.128052 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128022 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c63f860e-efee-41d4-8767-5169b323692d" (UID: "c63f860e-efee-41d4-8767-5169b323692d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:23.128147 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128055 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-console-config" (OuterVolumeSpecName: "console-config") pod "c63f860e-efee-41d4-8767-5169b323692d" (UID: "c63f860e-efee-41d4-8767-5169b323692d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:23.128147 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128107 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-oauth-config\") pod \"c63f860e-efee-41d4-8767-5169b323692d\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " Jul 16 17:17:23.128147 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128121 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-service-ca" (OuterVolumeSpecName: "service-ca") pod "c63f860e-efee-41d4-8767-5169b323692d" (UID: "c63f860e-efee-41d4-8767-5169b323692d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:23.128147 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128146 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-oauth-serving-cert\") pod \"c63f860e-efee-41d4-8767-5169b323692d\" (UID: \"c63f860e-efee-41d4-8767-5169b323692d\") " Jul 16 17:17:23.128436 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128414 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-service-ca\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:17:23.128576 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128543 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-trusted-ca-bundle\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:17:23.128576 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128564 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-console-config\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:17:23.128671 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.128628 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c63f860e-efee-41d4-8767-5169b323692d" (UID: "c63f860e-efee-41d4-8767-5169b323692d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:17:23.130530 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.130502 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c63f860e-efee-41d4-8767-5169b323692d" (UID: "c63f860e-efee-41d4-8767-5169b323692d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:17:23.130627 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.130585 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c63f860e-efee-41d4-8767-5169b323692d-kube-api-access-pc9m2" (OuterVolumeSpecName: "kube-api-access-pc9m2") pod "c63f860e-efee-41d4-8767-5169b323692d" (UID: "c63f860e-efee-41d4-8767-5169b323692d"). InnerVolumeSpecName "kube-api-access-pc9m2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:17:23.130668 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.130649 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c63f860e-efee-41d4-8767-5169b323692d" (UID: "c63f860e-efee-41d4-8767-5169b323692d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:17:23.229302 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.229225 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-serving-cert\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:17:23.229302 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.229250 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc9m2\" (UniqueName: \"kubernetes.io/projected/c63f860e-efee-41d4-8767-5169b323692d-kube-api-access-pc9m2\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:17:23.229302 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.229260 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c63f860e-efee-41d4-8767-5169b323692d-console-oauth-config\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:17:23.229302 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.229269 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c63f860e-efee-41d4-8767-5169b323692d-oauth-serving-cert\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:17:23.365133 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.365100 2577 generic.go:356] "Generic (PLEG): container finished" podID="c63f860e-efee-41d4-8767-5169b323692d" containerID="0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a" exitCode=0 Jul 16 17:17:23.365310 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.365165 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-757b895bf9-d7mk4" Jul 16 17:17:23.365310 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.365198 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-757b895bf9-d7mk4" event={"ID":"c63f860e-efee-41d4-8767-5169b323692d","Type":"ContainerDied","Data":"0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a"} Jul 16 17:17:23.365310 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.365288 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-757b895bf9-d7mk4" event={"ID":"c63f860e-efee-41d4-8767-5169b323692d","Type":"ContainerDied","Data":"b5018cc253c6c8bf173db419045410be9cfc0ef6763ffa3312913cdfb0aed908"} Jul 16 17:17:23.365477 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.365327 2577 scope.go:117] "RemoveContainer" containerID="0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a" Jul 16 17:17:23.367219 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.367192 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4h2x5" event={"ID":"eb84a249-ddf6-430e-bf26-2496fdf6b5bd","Type":"ContainerStarted","Data":"02514b68573d7ecd299effddc81ba90e637535d95f84262ad298801d88b59924"} Jul 16 17:17:23.367336 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.367242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4h2x5" event={"ID":"eb84a249-ddf6-430e-bf26-2496fdf6b5bd","Type":"ContainerStarted","Data":"fbc310ddafe12f5743e71bd972f5b7d0bf4fcc56052d7637ece4475c88639a94"} Jul 16 17:17:23.367336 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.367317 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4h2x5" Jul 16 17:17:23.374207 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.374193 2577 scope.go:117] "RemoveContainer" containerID="0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a" Jul 16 17:17:23.374458 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:17:23.374438 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a\": container with ID starting with 0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a not found: ID does not exist" containerID="0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a" Jul 16 17:17:23.374508 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.374465 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a"} err="failed to get container status \"0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a\": rpc error: code = NotFound desc = could not find container \"0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a\": container with ID starting with 0f5669204b12c33d37d67f63a00b7274674845abc51e9d55f9da195f6d6c385a not found: ID does not exist" Jul 16 17:17:23.397728 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.397683 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4h2x5" podStartSLOduration=129.037338065 podStartE2EDuration="2m11.397669356s" totalImagesPullingTime="2.360331291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:15:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:17:23.396670928 +0000 UTC m=+166.246177290" watchObservedRunningTime="2026-07-16 17:17:23.397669356 +0000 UTC m=+166.247175718" Jul 16 17:17:23.416837 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.416810 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-757b895bf9-d7mk4"] Jul 16 17:17:23.419593 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.419574 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-757b895bf9-d7mk4"] Jul 16 17:17:23.686107 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:23.686075 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c63f860e-efee-41d4-8767-5169b323692d" path="/var/lib/kubelet/pods/c63f860e-efee-41d4-8767-5169b323692d/volumes" Jul 16 17:17:29.136429 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:29.136350 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:29.369849 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:29.369815 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:33.373724 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:33.373690 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4h2x5" Jul 16 17:17:42.584893 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:42.584863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:42.671382 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:42.671352 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:50.227018 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:50.226984 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:50.313009 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:50.312978 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:17:59.748843 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:59.748780 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:17:59.764063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:17:59.764032 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:00.526968 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:00.526932 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:01.406600 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:01.406566 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:03.923374 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:03.923336 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:04.026860 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:04.026809 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:04.096986 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:04.096957 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:08.153508 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:08.153473 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:12.619223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:12.619195 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:12.945899 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:12.945861 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:13.694319 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:13.694287 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:18:13.694690 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:13.694562 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="alertmanager" containerID="cri-o://656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970" gracePeriod=120 Jul 16 17:18:13.694690 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:13.694617 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-metric" containerID="cri-o://3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290" gracePeriod=120 Jul 16 17:18:13.694690 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:13.694645 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-web" containerID="cri-o://d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f" gracePeriod=120 Jul 16 17:18:13.694857 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:13.694665 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy" containerID="cri-o://9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570" gracePeriod=120 Jul 16 17:18:13.694857 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:13.694723 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="prom-label-proxy" containerID="cri-o://6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2" gracePeriod=120 Jul 16 17:18:13.694857 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:13.694697 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="config-reloader" containerID="cri-o://b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a" gracePeriod=120 Jul 16 17:18:14.569118 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569088 2577 generic.go:356] "Generic (PLEG): container finished" podID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerID="6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2" exitCode=0 Jul 16 17:18:14.569118 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569111 2577 generic.go:356] "Generic (PLEG): container finished" podID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerID="3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290" exitCode=0 Jul 16 17:18:14.569118 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569117 2577 generic.go:356] "Generic (PLEG): container finished" podID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerID="9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570" exitCode=0 Jul 16 17:18:14.569118 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569123 2577 generic.go:356] "Generic (PLEG): container finished" podID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerID="b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a" exitCode=0 Jul 16 17:18:14.569118 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569128 2577 generic.go:356] "Generic (PLEG): container finished" podID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerID="656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970" exitCode=0 Jul 16 17:18:14.569409 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569166 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2"} Jul 16 17:18:14.569409 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569235 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290"} Jul 16 17:18:14.569409 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570"} Jul 16 17:18:14.569409 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569285 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a"} Jul 16 17:18:14.569409 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.569311 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970"} Jul 16 17:18:14.942437 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:14.942413 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.028127 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028092 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-trusted-ca-bundle\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028142 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-config-volume\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028198 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-tls-assets\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028232 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-main-tls\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028262 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-main-db\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028298 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028287 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028554 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028316 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-config-out\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028554 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028342 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-web-config\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.028655 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028561 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:15.028655 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028620 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:15.029584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028821 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqnll\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-kube-api-access-lqnll\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.029584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028880 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-cluster-tls-config\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.029584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028908 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-metrics-client-ca\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.029584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.028976 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.029584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.029024 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"d421f33f-6a4f-4086-8268-d2e802be27e7\" (UID: \"d421f33f-6a4f-4086-8268-d2e802be27e7\") " Jul 16 17:18:15.029584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.029297 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.029584 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.029318 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-alertmanager-main-db\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.030815 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.030730 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:15.031875 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.031814 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:15.032069 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.032037 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-config-volume" (OuterVolumeSpecName: "config-volume") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:15.033162 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.033134 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:15.033251 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.033198 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:15.033367 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.033343 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:15.033600 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.033577 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:15.033665 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.033580 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-config-out" (OuterVolumeSpecName: "config-out") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:15.035082 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.035060 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-kube-api-access-lqnll" (OuterVolumeSpecName: "kube-api-access-lqnll") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "kube-api-access-lqnll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:15.037546 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.037209 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:15.043362 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.043341 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-web-config" (OuterVolumeSpecName: "web-config") pod "d421f33f-6a4f-4086-8268-d2e802be27e7" (UID: "d421f33f-6a4f-4086-8268-d2e802be27e7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:15.130457 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130425 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130457 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130456 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d421f33f-6a4f-4086-8268-d2e802be27e7-config-out\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130467 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-web-config\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130476 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqnll\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-kube-api-access-lqnll\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130485 2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-cluster-tls-config\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130495 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d421f33f-6a4f-4086-8268-d2e802be27e7-metrics-client-ca\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130504 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130513 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130521 2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-config-volume\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130529 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d421f33f-6a4f-4086-8268-d2e802be27e7-tls-assets\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.130637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.130537 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d421f33f-6a4f-4086-8268-d2e802be27e7-secret-alertmanager-main-tls\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:15.575924 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.575840 2577 generic.go:356] "Generic (PLEG): container finished" podID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerID="d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f" exitCode=0 Jul 16 17:18:15.576049 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.575911 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f"} Jul 16 17:18:15.576049 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.575927 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 17:18:15.576049 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.576011 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d421f33f-6a4f-4086-8268-d2e802be27e7","Type":"ContainerDied","Data":"7af71f0d6b90dfd8ab6a251b383cbfdf328c75c673b31bfc5afe19368c0ea9b0"} Jul 16 17:18:15.576049 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.576045 2577 scope.go:117] "RemoveContainer" containerID="6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2" Jul 16 17:18:15.583979 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.583835 2577 scope.go:117] "RemoveContainer" containerID="3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290" Jul 16 17:18:15.590525 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.590509 2577 scope.go:117] "RemoveContainer" containerID="9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570" Jul 16 17:18:15.596912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.596897 2577 scope.go:117] "RemoveContainer" containerID="d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f" Jul 16 17:18:15.603104 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.603089 2577 scope.go:117] "RemoveContainer" containerID="b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a" Jul 16 17:18:15.609533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.609508 2577 scope.go:117] "RemoveContainer" containerID="656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970" Jul 16 17:18:15.613682 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.613663 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:18:15.617193 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.617173 2577 scope.go:117] "RemoveContainer" containerID="d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de" Jul 16 17:18:15.620877 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.620857 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 17:18:15.624383 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.624368 2577 scope.go:117] "RemoveContainer" containerID="6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2" Jul 16 17:18:15.624619 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:15.624600 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2\": container with ID starting with 6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2 not found: ID does not exist" containerID="6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2" Jul 16 17:18:15.624672 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.624627 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2"} err="failed to get container status \"6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2\": rpc error: code = NotFound desc = could not find container \"6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2\": container with ID starting with 6b299b95c60c94a5fd46cfed26c22215cc0f9a82a0377d1a5e5c3f70256cadc2 not found: ID does not exist" Jul 16 17:18:15.624672 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.624652 2577 scope.go:117] "RemoveContainer" containerID="3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290" Jul 16 17:18:15.624917 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:15.624899 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290\": container with ID starting with 3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290 not found: ID does not exist" containerID="3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290" Jul 16 17:18:15.624966 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.624924 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290"} err="failed to get container status \"3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290\": rpc error: code = NotFound desc = could not find container \"3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290\": container with ID starting with 3631af44db8e45afff2aced442b07143f5cf9e13e45b7ca67825c17b8c826290 not found: ID does not exist" Jul 16 17:18:15.624966 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.624940 2577 scope.go:117] "RemoveContainer" containerID="9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570" Jul 16 17:18:15.625153 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:15.625137 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570\": container with ID starting with 9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570 not found: ID does not exist" containerID="9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570" Jul 16 17:18:15.625187 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625160 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570"} err="failed to get container status \"9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570\": rpc error: code = NotFound desc = could not find container \"9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570\": container with ID starting with 9a7bd30c66c48ba5ac6acb03b7d0174f0e73d502d41fdfe4cc309c8a853ee570 not found: ID does not exist" Jul 16 17:18:15.625187 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625175 2577 scope.go:117] "RemoveContainer" containerID="d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f" Jul 16 17:18:15.625406 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:15.625390 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f\": container with ID starting with d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f not found: ID does not exist" containerID="d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f" Jul 16 17:18:15.625460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625417 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f"} err="failed to get container status \"d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f\": rpc error: code = NotFound desc = could not find container \"d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f\": container with ID starting with d128b542032bb9743c96a5581e09a2f6e03801f50d173953d850582a46cf644f not found: ID does not exist" Jul 16 17:18:15.625460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625432 2577 scope.go:117] "RemoveContainer" containerID="b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a" Jul 16 17:18:15.625660 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:15.625645 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a\": container with ID starting with b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a not found: ID does not exist" containerID="b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a" Jul 16 17:18:15.625698 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625662 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a"} err="failed to get container status \"b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a\": rpc error: code = NotFound desc = could not find container \"b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a\": container with ID starting with b1ac23327258376ede1c17109289d8a52424810d7bed2330e67e6d63d807893a not found: ID does not exist" Jul 16 17:18:15.625698 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625677 2577 scope.go:117] "RemoveContainer" containerID="656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970" Jul 16 17:18:15.625943 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:15.625929 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970\": container with ID starting with 656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970 not found: ID does not exist" containerID="656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970" Jul 16 17:18:15.625987 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625945 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970"} err="failed to get container status \"656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970\": rpc error: code = NotFound desc = could not find container \"656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970\": container with ID starting with 656ebc48c1a5f249dc741eb9461c9855bf3bd09643b69c68aac8cb3016642970 not found: ID does not exist" Jul 16 17:18:15.625987 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.625959 2577 scope.go:117] "RemoveContainer" containerID="d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de" Jul 16 17:18:15.626182 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:15.626165 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de\": container with ID starting with d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de not found: ID does not exist" containerID="d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de" Jul 16 17:18:15.626219 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.626186 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de"} err="failed to get container status \"d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de\": rpc error: code = NotFound desc = could not find container \"d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de\": container with ID starting with d465525c2a8f2a55e403904e1372efa360aad8af92596965e5dd3f991f3533de not found: ID does not exist" Jul 16 17:18:15.685890 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:15.685859 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" path="/var/lib/kubelet/pods/d421f33f-6a4f-4086-8268-d2e802be27e7/volumes" Jul 16 17:18:17.812637 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812604 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d"] Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812894 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c63f860e-efee-41d4-8767-5169b323692d" containerName="console" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812905 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c63f860e-efee-41d4-8767-5169b323692d" containerName="console" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812913 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="prom-label-proxy" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812920 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="prom-label-proxy" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812928 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-metric" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812934 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-metric" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812946 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="alertmanager" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812951 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="alertmanager" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812958 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="config-reloader" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812963 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="config-reloader" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812972 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-web" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812976 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-web" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812988 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812992 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.812998 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="init-config-reloader" Jul 16 17:18:17.813030 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813004 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="init-config-reloader" Jul 16 17:18:17.813485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813041 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="alertmanager" Jul 16 17:18:17.813485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813049 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="prom-label-proxy" Jul 16 17:18:17.813485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813055 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c63f860e-efee-41d4-8767-5169b323692d" containerName="console" Jul 16 17:18:17.813485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813063 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-metric" Jul 16 17:18:17.813485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813069 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="config-reloader" Jul 16 17:18:17.813485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813075 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy-web" Jul 16 17:18:17.813485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.813080 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d421f33f-6a4f-4086-8268-d2e802be27e7" containerName="kube-rbac-proxy" Jul 16 17:18:17.817617 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.817599 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.822974 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.822953 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 16 17:18:17.823090 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.823058 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 16 17:18:17.823090 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.823058 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 16 17:18:17.824141 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.824123 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-22mhm\"" Jul 16 17:18:17.824141 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.824136 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 16 17:18:17.825691 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.825674 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 16 17:18:17.828688 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.828671 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 16 17:18:17.836110 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.836091 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d"] Jul 16 17:18:17.955474 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.955474 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955476 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-serving-certs-ca-bundle\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.955697 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955500 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-telemeter-client-tls\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.955697 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955519 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-metrics-client-ca\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.955697 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955616 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-secret-telemeter-client\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.955697 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955645 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-federate-client-tls\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.955697 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:17.955960 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:17.955709 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94cf\" (UniqueName: \"kubernetes.io/projected/eade205c-7424-467c-9ffe-3a16ff098a52-kube-api-access-d94cf\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056319 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056290 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-telemeter-client-tls\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056319 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056326 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-metrics-client-ca\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-secret-telemeter-client\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-federate-client-tls\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d94cf\" (UniqueName: \"kubernetes.io/projected/eade205c-7424-467c-9ffe-3a16ff098a52-kube-api-access-d94cf\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056470 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.056533 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.056501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-serving-certs-ca-bundle\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.057169 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.057142 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-metrics-client-ca\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.057359 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.057165 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-serving-certs-ca-bundle\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.057482 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.057265 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eade205c-7424-467c-9ffe-3a16ff098a52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.059316 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.059291 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.059433 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.059414 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-secret-telemeter-client\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.059561 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.059539 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-telemeter-client-tls\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.059639 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.059540 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eade205c-7424-467c-9ffe-3a16ff098a52-federate-client-tls\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.076128 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.076073 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94cf\" (UniqueName: \"kubernetes.io/projected/eade205c-7424-467c-9ffe-3a16ff098a52-kube-api-access-d94cf\") pod \"telemeter-client-5b9c457dbb-5zf5d\" (UID: \"eade205c-7424-467c-9ffe-3a16ff098a52\") " pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.125897 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.125848 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" Jul 16 17:18:18.130685 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.130440 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:18:18.131051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.130853 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="prometheus" containerID="cri-o://0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" gracePeriod=600 Jul 16 17:18:18.131051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.130888 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy" containerID="cri-o://1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" gracePeriod=600 Jul 16 17:18:18.131051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.130880 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-thanos" containerID="cri-o://81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" gracePeriod=600 Jul 16 17:18:18.131051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.130915 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="thanos-sidecar" containerID="cri-o://6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" gracePeriod=600 Jul 16 17:18:18.131051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.130863 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-web" containerID="cri-o://3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" gracePeriod=600 Jul 16 17:18:18.131051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.131002 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="config-reloader" containerID="cri-o://b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" gracePeriod=600 Jul 16 17:18:18.323829 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.323723 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d"] Jul 16 17:18:18.350799 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:18:18.350761 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeade205c_7424_467c_9ffe_3a16ff098a52.slice/crio-4e971b03a2c8eb90e4244ce85ff3c954ba30a6044430b480dfe37022110ebf43 WatchSource:0}: Error finding container 4e971b03a2c8eb90e4244ce85ff3c954ba30a6044430b480dfe37022110ebf43: Status 404 returned error can't find the container with id 4e971b03a2c8eb90e4244ce85ff3c954ba30a6044430b480dfe37022110ebf43 Jul 16 17:18:18.470421 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.470133 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.561192 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561165 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-tls-assets\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561468 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561206 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28qz8\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-kube-api-access-28qz8\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561468 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561242 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-tls\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561468 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-1\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561468 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561383 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-metrics-client-ca\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561468 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561422 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561725 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561610 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-db\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561725 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561650 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561725 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561664 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:18.561725 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561683 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-serving-certs-ca-bundle\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561725 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561721 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-kube-rbac-proxy\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561750 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-web-config\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561785 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-0\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561856 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-trusted-ca-bundle\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561885 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config-out\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561926 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-grpc-tls\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561956 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-thanos-prometheus-http-client-file\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.561997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.561991 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-kubelet-serving-ca-bundle\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.562314 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562026 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-2\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.562314 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562052 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.562314 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562077 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-metrics-client-certs\") pod \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\" (UID: \"c4ab787f-9002-4b69-8236-d80a65ef1dfc\") " Jul 16 17:18:18.562314 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562114 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:18.562507 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562326 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.562507 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562344 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-metrics-client-ca\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.562679 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562651 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:18.563013 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.562977 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:18.563922 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.563647 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:18.565108 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.565077 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:18.565425 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.565340 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:18.565425 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.565347 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.565425 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.565354 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config-out" (OuterVolumeSpecName: "config-out") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 17:18:18.565425 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.565398 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.565957 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.565599 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config" (OuterVolumeSpecName: "config") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.565957 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.565640 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 17:18:18.566063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.566017 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-kube-api-access-28qz8" (OuterVolumeSpecName: "kube-api-access-28qz8") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "kube-api-access-28qz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:18.566063 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.566031 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 17:18:18.566165 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.566120 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.566367 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.566339 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.566900 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.566877 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.567319 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.567296 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.567379 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.567360 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.576492 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.576430 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-web-config" (OuterVolumeSpecName: "web-config") pod "c4ab787f-9002-4b69-8236-d80a65ef1dfc" (UID: "c4ab787f-9002-4b69-8236-d80a65ef1dfc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 17:18:18.588673 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.588646 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" event={"ID":"eade205c-7424-467c-9ffe-3a16ff098a52","Type":"ContainerStarted","Data":"4e971b03a2c8eb90e4244ce85ff3c954ba30a6044430b480dfe37022110ebf43"} Jul 16 17:18:18.591456 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591434 2577 generic.go:356] "Generic (PLEG): container finished" podID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" exitCode=0 Jul 16 17:18:18.591456 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591454 2577 generic.go:356] "Generic (PLEG): container finished" podID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" exitCode=0 Jul 16 17:18:18.591566 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591462 2577 generic.go:356] "Generic (PLEG): container finished" podID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" exitCode=0 Jul 16 17:18:18.591566 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591468 2577 generic.go:356] "Generic (PLEG): container finished" podID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" exitCode=0 Jul 16 17:18:18.591566 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591472 2577 generic.go:356] "Generic (PLEG): container finished" podID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" exitCode=0 Jul 16 17:18:18.591566 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591477 2577 generic.go:356] "Generic (PLEG): container finished" podID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" exitCode=0 Jul 16 17:18:18.591566 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591505 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.591566 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591520 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} Jul 16 17:18:18.591733 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591588 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} Jul 16 17:18:18.591733 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591616 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} Jul 16 17:18:18.591733 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591641 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} Jul 16 17:18:18.591733 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591656 2577 scope.go:117] "RemoveContainer" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.591733 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591679 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} Jul 16 17:18:18.591733 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591719 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} Jul 16 17:18:18.592007 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.591757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c4ab787f-9002-4b69-8236-d80a65ef1dfc","Type":"ContainerDied","Data":"0269006cf0348ac2249bdedb5281c21cb5def9a7f9cd5d6c2669d5d41bab2b3e"} Jul 16 17:18:18.599983 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.599738 2577 scope.go:117] "RemoveContainer" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.607555 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.607538 2577 scope.go:117] "RemoveContainer" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.613835 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.613817 2577 scope.go:117] "RemoveContainer" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.620741 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.620720 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:18:18.620912 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.620897 2577 scope.go:117] "RemoveContainer" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.626325 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.626304 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:18:18.627557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.627542 2577 scope.go:117] "RemoveContainer" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.633999 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.633985 2577 scope.go:117] "RemoveContainer" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.640543 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.640525 2577 scope.go:117] "RemoveContainer" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.640765 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:18.640750 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": container with ID starting with 81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc not found: ID does not exist" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.640838 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.640772 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} err="failed to get container status \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": rpc error: code = NotFound desc = could not find container \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": container with ID starting with 81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc not found: ID does not exist" Jul 16 17:18:18.640838 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.640802 2577 scope.go:117] "RemoveContainer" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.641034 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:18.641017 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": container with ID starting with 1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a not found: ID does not exist" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.641080 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641040 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} err="failed to get container status \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": rpc error: code = NotFound desc = could not find container \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": container with ID starting with 1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a not found: ID does not exist" Jul 16 17:18:18.641080 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641056 2577 scope.go:117] "RemoveContainer" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.641282 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:18.641265 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": container with ID starting with 3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f not found: ID does not exist" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.641342 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641289 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} err="failed to get container status \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": rpc error: code = NotFound desc = could not find container \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": container with ID starting with 3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f not found: ID does not exist" Jul 16 17:18:18.641342 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641312 2577 scope.go:117] "RemoveContainer" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.641557 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:18.641540 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": container with ID starting with 6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3 not found: ID does not exist" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.641591 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641560 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} err="failed to get container status \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": rpc error: code = NotFound desc = could not find container \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": container with ID starting with 6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3 not found: ID does not exist" Jul 16 17:18:18.641591 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641573 2577 scope.go:117] "RemoveContainer" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.641773 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:18.641761 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": container with ID starting with b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4 not found: ID does not exist" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.641869 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641775 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} err="failed to get container status \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": rpc error: code = NotFound desc = could not find container \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": container with ID starting with b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4 not found: ID does not exist" Jul 16 17:18:18.641869 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.641785 2577 scope.go:117] "RemoveContainer" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.642018 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:18.642002 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": container with ID starting with 0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7 not found: ID does not exist" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.642058 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642022 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} err="failed to get container status \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": rpc error: code = NotFound desc = could not find container \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": container with ID starting with 0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7 not found: ID does not exist" Jul 16 17:18:18.642058 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642044 2577 scope.go:117] "RemoveContainer" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.642277 ip-10-0-133-160 kubenswrapper[2577]: E0716 17:18:18.642258 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": container with ID starting with cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882 not found: ID does not exist" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.642343 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642282 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882"} err="failed to get container status \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": rpc error: code = NotFound desc = could not find container \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": container with ID starting with cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882 not found: ID does not exist" Jul 16 17:18:18.642343 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642295 2577 scope.go:117] "RemoveContainer" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.642534 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642518 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} err="failed to get container status \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": rpc error: code = NotFound desc = could not find container \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": container with ID starting with 81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc not found: ID does not exist" Jul 16 17:18:18.642571 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642536 2577 scope.go:117] "RemoveContainer" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.642744 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642728 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} err="failed to get container status \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": rpc error: code = NotFound desc = could not find container \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": container with ID starting with 1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a not found: ID does not exist" Jul 16 17:18:18.642778 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.642745 2577 scope.go:117] "RemoveContainer" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.643020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643003 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} err="failed to get container status \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": rpc error: code = NotFound desc = could not find container \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": container with ID starting with 3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f not found: ID does not exist" Jul 16 17:18:18.643058 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643022 2577 scope.go:117] "RemoveContainer" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.643223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643207 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} err="failed to get container status \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": rpc error: code = NotFound desc = could not find container \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": container with ID starting with 6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3 not found: ID does not exist" Jul 16 17:18:18.643258 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643225 2577 scope.go:117] "RemoveContainer" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.643417 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643398 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} err="failed to get container status \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": rpc error: code = NotFound desc = could not find container \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": container with ID starting with b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4 not found: ID does not exist" Jul 16 17:18:18.643457 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643418 2577 scope.go:117] "RemoveContainer" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.643598 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643580 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} err="failed to get container status \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": rpc error: code = NotFound desc = could not find container \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": container with ID starting with 0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7 not found: ID does not exist" Jul 16 17:18:18.643636 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643598 2577 scope.go:117] "RemoveContainer" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.643778 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643758 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882"} err="failed to get container status \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": rpc error: code = NotFound desc = could not find container \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": container with ID starting with cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882 not found: ID does not exist" Jul 16 17:18:18.643873 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643780 2577 scope.go:117] "RemoveContainer" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.643997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643978 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} err="failed to get container status \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": rpc error: code = NotFound desc = could not find container \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": container with ID starting with 81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc not found: ID does not exist" Jul 16 17:18:18.644039 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.643998 2577 scope.go:117] "RemoveContainer" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.644155 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644137 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} err="failed to get container status \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": rpc error: code = NotFound desc = could not find container \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": container with ID starting with 1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a not found: ID does not exist" Jul 16 17:18:18.644203 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644156 2577 scope.go:117] "RemoveContainer" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.644370 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644354 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} err="failed to get container status \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": rpc error: code = NotFound desc = could not find container \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": container with ID starting with 3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f not found: ID does not exist" Jul 16 17:18:18.644370 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644370 2577 scope.go:117] "RemoveContainer" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.644573 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644556 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} err="failed to get container status \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": rpc error: code = NotFound desc = could not find container \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": container with ID starting with 6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3 not found: ID does not exist" Jul 16 17:18:18.644631 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644576 2577 scope.go:117] "RemoveContainer" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.644820 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644773 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} err="failed to get container status \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": rpc error: code = NotFound desc = could not find container \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": container with ID starting with b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4 not found: ID does not exist" Jul 16 17:18:18.644872 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.644836 2577 scope.go:117] "RemoveContainer" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.645038 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645021 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} err="failed to get container status \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": rpc error: code = NotFound desc = could not find container \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": container with ID starting with 0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7 not found: ID does not exist" Jul 16 17:18:18.645081 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645039 2577 scope.go:117] "RemoveContainer" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.645233 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645219 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882"} err="failed to get container status \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": rpc error: code = NotFound desc = could not find container \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": container with ID starting with cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882 not found: ID does not exist" Jul 16 17:18:18.645274 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645233 2577 scope.go:117] "RemoveContainer" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.645423 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645406 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} err="failed to get container status \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": rpc error: code = NotFound desc = could not find container \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": container with ID starting with 81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc not found: ID does not exist" Jul 16 17:18:18.645468 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645425 2577 scope.go:117] "RemoveContainer" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.645597 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645583 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} err="failed to get container status \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": rpc error: code = NotFound desc = could not find container \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": container with ID starting with 1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a not found: ID does not exist" Jul 16 17:18:18.645641 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645596 2577 scope.go:117] "RemoveContainer" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.645827 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645783 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} err="failed to get container status \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": rpc error: code = NotFound desc = could not find container \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": container with ID starting with 3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f not found: ID does not exist" Jul 16 17:18:18.645827 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.645826 2577 scope.go:117] "RemoveContainer" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.646026 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646010 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} err="failed to get container status \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": rpc error: code = NotFound desc = could not find container \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": container with ID starting with 6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3 not found: ID does not exist" Jul 16 17:18:18.646074 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646026 2577 scope.go:117] "RemoveContainer" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.646226 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646210 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} err="failed to get container status \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": rpc error: code = NotFound desc = could not find container \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": container with ID starting with b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4 not found: ID does not exist" Jul 16 17:18:18.646291 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646227 2577 scope.go:117] "RemoveContainer" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.646416 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646401 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} err="failed to get container status \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": rpc error: code = NotFound desc = could not find container \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": container with ID starting with 0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7 not found: ID does not exist" Jul 16 17:18:18.646458 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646416 2577 scope.go:117] "RemoveContainer" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.646610 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646591 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882"} err="failed to get container status \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": rpc error: code = NotFound desc = could not find container \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": container with ID starting with cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882 not found: ID does not exist" Jul 16 17:18:18.646690 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646611 2577 scope.go:117] "RemoveContainer" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.646811 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646779 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} err="failed to get container status \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": rpc error: code = NotFound desc = could not find container \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": container with ID starting with 81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc not found: ID does not exist" Jul 16 17:18:18.646854 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.646813 2577 scope.go:117] "RemoveContainer" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.647018 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647002 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} err="failed to get container status \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": rpc error: code = NotFound desc = could not find container \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": container with ID starting with 1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a not found: ID does not exist" Jul 16 17:18:18.647066 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647018 2577 scope.go:117] "RemoveContainer" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.647200 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647185 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} err="failed to get container status \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": rpc error: code = NotFound desc = could not find container \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": container with ID starting with 3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f not found: ID does not exist" Jul 16 17:18:18.647242 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647201 2577 scope.go:117] "RemoveContainer" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.647379 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647360 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} err="failed to get container status \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": rpc error: code = NotFound desc = could not find container \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": container with ID starting with 6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3 not found: ID does not exist" Jul 16 17:18:18.647419 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647379 2577 scope.go:117] "RemoveContainer" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.647546 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647525 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} err="failed to get container status \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": rpc error: code = NotFound desc = could not find container \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": container with ID starting with b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4 not found: ID does not exist" Jul 16 17:18:18.647626 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647549 2577 scope.go:117] "RemoveContainer" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.647761 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647746 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} err="failed to get container status \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": rpc error: code = NotFound desc = could not find container \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": container with ID starting with 0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7 not found: ID does not exist" Jul 16 17:18:18.647824 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647762 2577 scope.go:117] "RemoveContainer" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.647969 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647952 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882"} err="failed to get container status \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": rpc error: code = NotFound desc = could not find container \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": container with ID starting with cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882 not found: ID does not exist" Jul 16 17:18:18.648033 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.647971 2577 scope.go:117] "RemoveContainer" containerID="81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc" Jul 16 17:18:18.648159 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648145 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc"} err="failed to get container status \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": rpc error: code = NotFound desc = could not find container \"81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc\": container with ID starting with 81c2ffc3c62b7ce16d072b69fc5bb91aee511a47656ff6fd933dada9e4ea73bc not found: ID does not exist" Jul 16 17:18:18.648220 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648159 2577 scope.go:117] "RemoveContainer" containerID="1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a" Jul 16 17:18:18.648333 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648317 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a"} err="failed to get container status \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": rpc error: code = NotFound desc = could not find container \"1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a\": container with ID starting with 1df2b4b9094e99c78a166397dbe98121de999f18e33b539cd60b0f5be93f4a7a not found: ID does not exist" Jul 16 17:18:18.648375 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648334 2577 scope.go:117] "RemoveContainer" containerID="3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f" Jul 16 17:18:18.648511 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648495 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f"} err="failed to get container status \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": rpc error: code = NotFound desc = could not find container \"3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f\": container with ID starting with 3b404f19bed27917542b2d229ef85db7fd38c8083d2c9fe18165bca86f752a5f not found: ID does not exist" Jul 16 17:18:18.648550 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648513 2577 scope.go:117] "RemoveContainer" containerID="6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3" Jul 16 17:18:18.648677 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648657 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3"} err="failed to get container status \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": rpc error: code = NotFound desc = could not find container \"6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3\": container with ID starting with 6ef65a525619f988f0e2e73b49e35dc61457e0fc56d5c34090199f8b7d0066a3 not found: ID does not exist" Jul 16 17:18:18.648677 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648676 2577 scope.go:117] "RemoveContainer" containerID="b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4" Jul 16 17:18:18.648918 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648902 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4"} err="failed to get container status \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": rpc error: code = NotFound desc = could not find container \"b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4\": container with ID starting with b01fd1823f35e9cf9b9313826db347abc06c6345d209fdfbb36d2f437050b3b4 not found: ID does not exist" Jul 16 17:18:18.648918 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.648918 2577 scope.go:117] "RemoveContainer" containerID="0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7" Jul 16 17:18:18.649114 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.649096 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7"} err="failed to get container status \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": rpc error: code = NotFound desc = could not find container \"0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7\": container with ID starting with 0db20753f4a74b8991356d5063c7a195e2507bcf72b96c438642c4870a1049e7 not found: ID does not exist" Jul 16 17:18:18.649114 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.649113 2577 scope.go:117] "RemoveContainer" containerID="cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882" Jul 16 17:18:18.649328 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.649313 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882"} err="failed to get container status \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": rpc error: code = NotFound desc = could not find container \"cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882\": container with ID starting with cd82aafb5030863c218d1792fbd061e45052430c7a6c6e3361278f3b30f88882 not found: ID does not exist" Jul 16 17:18:18.662655 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662631 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662655 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662650 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662660 2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-kube-rbac-proxy\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662672 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-web-config\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662680 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662689 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662697 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config-out\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662705 2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-grpc-tls\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662712 2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662720 2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662729 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662737 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662747 2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-metrics-client-certs\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662755 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-tls-assets\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662762 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28qz8\" (UniqueName: \"kubernetes.io/projected/c4ab787f-9002-4b69-8236-d80a65ef1dfc-kube-api-access-28qz8\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662770 2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.662772 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662777 2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c4ab787f-9002-4b69-8236-d80a65ef1dfc-config\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.663201 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.662785 2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c4ab787f-9002-4b69-8236-d80a65ef1dfc-prometheus-k8s-db\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 17:18:18.667086 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667068 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:18:18.667324 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667313 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="config-reloader" Jul 16 17:18:18.667366 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667325 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="config-reloader" Jul 16 17:18:18.667366 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667334 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="init-config-reloader" Jul 16 17:18:18.667366 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667340 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="init-config-reloader" Jul 16 17:18:18.667366 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667346 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="prometheus" Jul 16 17:18:18.667366 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667351 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="prometheus" Jul 16 17:18:18.667366 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667358 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-web" Jul 16 17:18:18.667366 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667364 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-web" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667375 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-thanos" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667381 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-thanos" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667387 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667392 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667400 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="thanos-sidecar" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667405 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="thanos-sidecar" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667449 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="config-reloader" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667457 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-web" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667462 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy-thanos" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667468 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="prometheus" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667475 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="thanos-sidecar" Jul 16 17:18:18.667557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.667480 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" containerName="kube-rbac-proxy" Jul 16 17:18:18.672292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.672276 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.676460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.676445 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 16 17:18:18.676833 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.676804 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 16 17:18:18.676940 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.676863 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 16 17:18:18.676940 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.676913 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 16 17:18:18.677046 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.676968 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 16 17:18:18.677046 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.676911 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 16 17:18:18.677296 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.677281 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 16 17:18:18.677504 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.677413 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6e9mgf33es629\"" Jul 16 17:18:18.677631 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.677614 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 16 17:18:18.677902 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.677861 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 16 17:18:18.678010 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.677915 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 16 17:18:18.678107 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.678008 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 17:18:18.678426 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.678407 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vk8jr\"" Jul 16 17:18:18.678552 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.678424 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 16 17:18:18.680281 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.680265 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 16 17:18:18.681066 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.681045 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 16 17:18:18.682573 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.682556 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 16 17:18:18.692184 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.692164 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:18:18.763676 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763643 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.763676 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763678 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.763904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.763904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.763904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763787 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.763904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763837 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.763904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-config\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763937 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9w25\" (UniqueName: \"kubernetes.io/projected/c8914c5b-dd02-4307-ab75-84b8f6103bfe-kube-api-access-h9w25\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8914c5b-dd02-4307-ab75-84b8f6103bfe-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.763994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764020 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764124 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764161 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8914c5b-dd02-4307-ab75-84b8f6103bfe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.764315 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.764181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.864571 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.864571 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864527 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.864571 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864551 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.864571 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864568 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864685 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-config\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864762 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9w25\" (UniqueName: \"kubernetes.io/projected/c8914c5b-dd02-4307-ab75-84b8f6103bfe-kube-api-access-h9w25\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8914c5b-dd02-4307-ab75-84b8f6103bfe-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.864996 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865080 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865109 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8914c5b-dd02-4307-ab75-84b8f6103bfe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865812 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865812 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865265 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865812 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865812 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865411 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865812 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865594 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.865812 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865750 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.866111 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.865875 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.866324 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.866303 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.866739 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.866714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.867902 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.867741 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.867902 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.867758 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-config\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.867902 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.867819 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.868278 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.868227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.868648 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.868552 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.873951 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.869284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.873951 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.869535 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.873951 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.869696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-web-config\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.873951 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.870064 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c8914c5b-dd02-4307-ab75-84b8f6103bfe-config-out\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.873951 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.870190 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8914c5b-dd02-4307-ab75-84b8f6103bfe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.873951 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.870224 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.873951 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.870574 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c8914c5b-dd02-4307-ab75-84b8f6103bfe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.874574 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.874151 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c8914c5b-dd02-4307-ab75-84b8f6103bfe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.880202 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.880178 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9w25\" (UniqueName: \"kubernetes.io/projected/c8914c5b-dd02-4307-ab75-84b8f6103bfe-kube-api-access-h9w25\") pod \"prometheus-k8s-0\" (UID: \"c8914c5b-dd02-4307-ab75-84b8f6103bfe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:18.981847 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:18.981816 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:19.144780 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:19.144731 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 16 17:18:19.146611 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:18:19.146579 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8914c5b_dd02_4307_ab75_84b8f6103bfe.slice/crio-eeaaa534912118e7b58d14a2b4c4428db96e6687e32d6fc494131521c94447a4 WatchSource:0}: Error finding container eeaaa534912118e7b58d14a2b4c4428db96e6687e32d6fc494131521c94447a4: Status 404 returned error can't find the container with id eeaaa534912118e7b58d14a2b4c4428db96e6687e32d6fc494131521c94447a4 Jul 16 17:18:19.597613 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:19.597527 2577 generic.go:356] "Generic (PLEG): container finished" podID="c8914c5b-dd02-4307-ab75-84b8f6103bfe" containerID="cf2da8cb16ea527ef725203d4717a99a834870d3333ccd1cec9f35520ff67e76" exitCode=0 Jul 16 17:18:19.597613 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:19.597565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerDied","Data":"cf2da8cb16ea527ef725203d4717a99a834870d3333ccd1cec9f35520ff67e76"} Jul 16 17:18:19.597845 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:19.597630 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerStarted","Data":"eeaaa534912118e7b58d14a2b4c4428db96e6687e32d6fc494131521c94447a4"} Jul 16 17:18:19.689132 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:19.689099 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4ab787f-9002-4b69-8236-d80a65ef1dfc" path="/var/lib/kubelet/pods/c4ab787f-9002-4b69-8236-d80a65ef1dfc/volumes" Jul 16 17:18:20.604496 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.604460 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" event={"ID":"eade205c-7424-467c-9ffe-3a16ff098a52","Type":"ContainerStarted","Data":"45901d65605569055dde499502f5b0034982ae76d9f6265a62ef7bd23a6288cf"} Jul 16 17:18:20.608308 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.608284 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerStarted","Data":"3c5f71f5880f85c6dabae0e69869caf768e4524efabc2ddc90c838d4cc74e127"} Jul 16 17:18:20.608412 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.608343 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerStarted","Data":"e85e609c43ee37e6ee51a34d2f02bb9705724c9aef77b2304a8b3511d5d79799"} Jul 16 17:18:20.608412 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.608380 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerStarted","Data":"f6e03eddaafeb3f36500dddbf8c697fe473c6dd52cecfb9f3a90e5464425d32d"} Jul 16 17:18:20.608412 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.608409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerStarted","Data":"e823496ad930c6a0ff3e33ee7986ecc619a99e705d28018c03e9e012639a0d5b"} Jul 16 17:18:20.608570 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.608433 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerStarted","Data":"74241bfff06f46fe7efd098109ac656ea33358faa683a17efb8b1847964bed00"} Jul 16 17:18:20.608570 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.608469 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c8914c5b-dd02-4307-ab75-84b8f6103bfe","Type":"ContainerStarted","Data":"86abe00d1d4bd4c29c33ffea046e30f5310760577eaa98371388d4d9b3aa0c55"} Jul 16 17:18:20.643927 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:20.643883 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.624580239 podStartE2EDuration="2.643864738s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.284499ms" isStatefulPod=false podCreationTimestamp="2026-07-16 17:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:18:20.642290365 +0000 UTC m=+223.491796727" watchObservedRunningTime="2026-07-16 17:18:20.643864738 +0000 UTC m=+223.493371098" Jul 16 17:18:21.613902 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:21.613859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" event={"ID":"eade205c-7424-467c-9ffe-3a16ff098a52","Type":"ContainerStarted","Data":"481647427cfd96bf616e49a4bb70ffb824a0ca928e60e19f3bcd5bd6eb5fbe93"} Jul 16 17:18:21.614283 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:21.613970 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" event={"ID":"eade205c-7424-467c-9ffe-3a16ff098a52","Type":"ContainerStarted","Data":"ff0ad32414ecb2a8b2f2c6f18718cc12a567820bc0b2b7a95a4546e2130c04d8"} Jul 16 17:18:21.645953 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:21.645897 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5b9c457dbb-5zf5d" podStartSLOduration=2.472435628 podStartE2EDuration="4.645880342s" totalImagesPullingTime="2.173444714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:18:21.64568657 +0000 UTC m=+224.495192930" watchObservedRunningTime="2026-07-16 17:18:21.645880342 +0000 UTC m=+224.495386702" Jul 16 17:18:22.776008 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:22.775968 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:22.851131 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:22.851099 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:23.982446 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:23.982412 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:18:30.416771 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:30.416737 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:30.499656 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:30.499623 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:33.547672 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:33.547638 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:18:33.636485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:18:33.636452 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:18.982727 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:18.982689 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:19:18.997824 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:18.997785 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:19:19.841767 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:19.841740 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 16 17:19:27.989495 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:27.989463 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:28.090114 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:28.090087 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:29.349714 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:29.349677 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:29.420237 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:29.420207 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:37.529211 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:37.529176 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:19:37.532409 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:37.532388 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 17:19:37.532878 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:37.532864 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:19:39.952738 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:39.952699 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:40.024327 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:40.024297 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:40.805513 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:40.805478 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:40.908854 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:40.908825 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:40.984238 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:40.984208 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:51.527009 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:51.526970 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:19:51.624983 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:19:51.624949 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:20:57.545350 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:20:57.545317 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:00.175878 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:00.175847 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:20.646863 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:20.646821 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:20.744603 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:20.744569 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:26.524384 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:26.524352 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:26.826131 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:26.826032 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:27.044278 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:27.044246 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:27.183966 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:27.183928 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:27.284371 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:27.284338 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:27.455369 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:27.455278 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:27.557152 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:27.557115 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:27.653742 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:27.653705 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:27.991231 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:27.991194 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:28.293718 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:28.293637 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:28.389076 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:28.389042 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:32.451138 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:32.451100 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:21:32.555514 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:21:32.555481 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:08.062899 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:08.062823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:14.200459 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:14.200430 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:14.301243 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:14.301217 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:14.751131 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:14.751103 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:36.493018 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:36.492982 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:36.595624 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:36.595599 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:36.761603 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:36.761528 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:36.858612 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:36.858584 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:36.964103 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:36.964070 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:37.738521 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:37.738490 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:37.839372 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:37.839342 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:37.942692 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:37.942662 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:45.626630 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:45.626597 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:47.792410 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:47.792367 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:22:47.864336 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:22:47.864311 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:21.999805 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:21.999742 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:22.094203 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:22.094171 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:29.968836 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:29.968712 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:33.241013 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:33.240974 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:34.131418 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:34.131388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:36.858101 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:36.858070 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:36.957172 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:36.957137 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:40.565433 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.565398 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:40.657719 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.657683 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:40.741807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.741772 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j9ch2"] Jul 16 17:23:40.745133 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.745112 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:40.748264 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.748243 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 17:23:40.748428 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.748279 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-d4vr5\"" Jul 16 17:23:40.748522 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.748508 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 17:23:40.754950 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.754921 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j9ch2"] Jul 16 17:23:40.870501 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.870476 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:40.907156 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.907107 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6b4b50e-3aec-4fa2-8198-7499423bc661-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j9ch2\" (UID: \"c6b4b50e-3aec-4fa2-8198-7499423bc661\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:40.907338 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.907163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ldds\" (UniqueName: \"kubernetes.io/projected/c6b4b50e-3aec-4fa2-8198-7499423bc661-kube-api-access-2ldds\") pod \"cert-manager-webhook-6486ffd997-j9ch2\" (UID: \"c6b4b50e-3aec-4fa2-8198-7499423bc661\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:40.978857 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:40.978817 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:41.008078 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.008038 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6b4b50e-3aec-4fa2-8198-7499423bc661-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j9ch2\" (UID: \"c6b4b50e-3aec-4fa2-8198-7499423bc661\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:41.008078 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.008082 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ldds\" (UniqueName: \"kubernetes.io/projected/c6b4b50e-3aec-4fa2-8198-7499423bc661-kube-api-access-2ldds\") pod \"cert-manager-webhook-6486ffd997-j9ch2\" (UID: \"c6b4b50e-3aec-4fa2-8198-7499423bc661\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:41.028365 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.028332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6b4b50e-3aec-4fa2-8198-7499423bc661-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j9ch2\" (UID: \"c6b4b50e-3aec-4fa2-8198-7499423bc661\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:41.028618 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.028596 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ldds\" (UniqueName: \"kubernetes.io/projected/c6b4b50e-3aec-4fa2-8198-7499423bc661-kube-api-access-2ldds\") pod \"cert-manager-webhook-6486ffd997-j9ch2\" (UID: \"c6b4b50e-3aec-4fa2-8198-7499423bc661\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:41.062303 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.062268 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:41.077157 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.077127 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:41.200941 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.200918 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j9ch2"] Jul 16 17:23:41.203078 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:23:41.203052 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6b4b50e_3aec_4fa2_8198_7499423bc661.slice/crio-4ac03dc9b3ca3bf62c5233851d689f5c32af7ee34ad59a7f85f422d77453d0e5 WatchSource:0}: Error finding container 4ac03dc9b3ca3bf62c5233851d689f5c32af7ee34ad59a7f85f422d77453d0e5: Status 404 returned error can't find the container with id 4ac03dc9b3ca3bf62c5233851d689f5c32af7ee34ad59a7f85f422d77453d0e5 Jul 16 17:23:41.204959 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.204944 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:23:41.764640 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.764604 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" event={"ID":"c6b4b50e-3aec-4fa2-8198-7499423bc661","Type":"ContainerStarted","Data":"4ac03dc9b3ca3bf62c5233851d689f5c32af7ee34ad59a7f85f422d77453d0e5"} Jul 16 17:23:41.846751 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.846719 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:41.950052 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:41.950024 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:42.780782 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:42.780706 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:43.177966 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:43.177926 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:43.569051 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:43.568951 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:43.668089 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:43.668058 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:43.768293 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:43.768259 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:44.779380 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:44.779336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" event={"ID":"c6b4b50e-3aec-4fa2-8198-7499423bc661","Type":"ContainerStarted","Data":"212ed8dd3edf07570ab470ca5780311994f30180af2d66013aa11d30ffdf4f92"} Jul 16 17:23:44.779783 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:44.779468 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:44.800607 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:44.800559 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" podStartSLOduration=1.646098636 podStartE2EDuration="4.80054405s" totalImagesPullingTime="3.154445414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:23:44.799088344 +0000 UTC m=+547.648594705" watchObservedRunningTime="2026-07-16 17:23:44.80054405 +0000 UTC m=+547.650050411" Jul 16 17:23:47.175302 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.175262 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wh98s"] Jul 16 17:23:47.178605 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.178582 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.181653 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.181628 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wdtmj\"" Jul 16 17:23:47.191008 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.190984 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wh98s"] Jul 16 17:23:47.259610 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.259573 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxlb2\" (UniqueName: \"kubernetes.io/projected/56e0725a-8eab-406e-9f23-91a053199864-kube-api-access-pxlb2\") pod \"cert-manager-cainjector-559cdd8c87-wh98s\" (UID: \"56e0725a-8eab-406e-9f23-91a053199864\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.259759 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.259637 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/56e0725a-8eab-406e-9f23-91a053199864-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wh98s\" (UID: \"56e0725a-8eab-406e-9f23-91a053199864\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.360986 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.360946 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxlb2\" (UniqueName: \"kubernetes.io/projected/56e0725a-8eab-406e-9f23-91a053199864-kube-api-access-pxlb2\") pod \"cert-manager-cainjector-559cdd8c87-wh98s\" (UID: \"56e0725a-8eab-406e-9f23-91a053199864\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.361164 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.361052 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/56e0725a-8eab-406e-9f23-91a053199864-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wh98s\" (UID: \"56e0725a-8eab-406e-9f23-91a053199864\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.375807 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.375770 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/56e0725a-8eab-406e-9f23-91a053199864-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wh98s\" (UID: \"56e0725a-8eab-406e-9f23-91a053199864\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.375959 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.375855 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxlb2\" (UniqueName: \"kubernetes.io/projected/56e0725a-8eab-406e-9f23-91a053199864-kube-api-access-pxlb2\") pod \"cert-manager-cainjector-559cdd8c87-wh98s\" (UID: \"56e0725a-8eab-406e-9f23-91a053199864\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.488165 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.488085 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" Jul 16 17:23:47.620554 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.620528 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wh98s"] Jul 16 17:23:47.622875 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:23:47.622847 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56e0725a_8eab_406e_9f23_91a053199864.slice/crio-e44c7fe42c5da7c78bce1b50a602f69ae38be3015a84a213c0e1cfb356546a18 WatchSource:0}: Error finding container e44c7fe42c5da7c78bce1b50a602f69ae38be3015a84a213c0e1cfb356546a18: Status 404 returned error can't find the container with id e44c7fe42c5da7c78bce1b50a602f69ae38be3015a84a213c0e1cfb356546a18 Jul 16 17:23:47.791897 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.791804 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" event={"ID":"56e0725a-8eab-406e-9f23-91a053199864","Type":"ContainerStarted","Data":"7e4a5cc6c98fe14168afff041f652899cf58c229637e871abc23beae96f68548"} Jul 16 17:23:47.791897 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.791846 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" event={"ID":"56e0725a-8eab-406e-9f23-91a053199864","Type":"ContainerStarted","Data":"e44c7fe42c5da7c78bce1b50a602f69ae38be3015a84a213c0e1cfb356546a18"} Jul 16 17:23:47.816359 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:47.816307 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wh98s" podStartSLOduration=0.81628903 podStartE2EDuration="816.28903ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:23:47.815193939 +0000 UTC m=+550.664700302" watchObservedRunningTime="2026-07-16 17:23:47.81628903 +0000 UTC m=+550.665795392" Jul 16 17:23:48.635141 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:48.635097 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:48.715943 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:48.715909 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:50.785402 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:50.785366 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j9ch2" Jul 16 17:23:51.404245 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:51.404219 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:55.630848 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:55.630817 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:23:55.751685 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:23:55.751655 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:07.401965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:07.401914 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:11.599091 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.599056 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:11.695723 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.695690 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:11.786087 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.786053 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf"] Jul 16 17:24:11.789440 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.789419 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.792497 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.792471 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 17:24:11.792784 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.792768 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 17:24:11.794001 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.793973 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 17:24:11.794109 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.794036 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-65mct\"" Jul 16 17:24:11.794109 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.794077 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 17:24:11.796210 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.796159 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 17:24:11.807970 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.807943 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf"] Jul 16 17:24:11.863011 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.862929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7n5v\" (UniqueName: \"kubernetes.io/projected/5dd0572f-2478-4da0-b0ec-56cb53404123-kube-api-access-g7n5v\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.863011 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.862977 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5dd0572f-2478-4da0-b0ec-56cb53404123-manager-config\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.863011 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.863003 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dd0572f-2478-4da0-b0ec-56cb53404123-cert\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.863222 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.863041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5dd0572f-2478-4da0-b0ec-56cb53404123-metrics-cert\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.963901 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.963869 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7n5v\" (UniqueName: \"kubernetes.io/projected/5dd0572f-2478-4da0-b0ec-56cb53404123-kube-api-access-g7n5v\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.964096 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.963924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5dd0572f-2478-4da0-b0ec-56cb53404123-manager-config\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.964096 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.963946 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dd0572f-2478-4da0-b0ec-56cb53404123-cert\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.964096 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.963971 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5dd0572f-2478-4da0-b0ec-56cb53404123-metrics-cert\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.964607 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.964585 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5dd0572f-2478-4da0-b0ec-56cb53404123-manager-config\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.966485 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.966465 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5dd0572f-2478-4da0-b0ec-56cb53404123-metrics-cert\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.966580 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.966526 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dd0572f-2478-4da0-b0ec-56cb53404123-cert\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:11.979846 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:11.979818 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7n5v\" (UniqueName: \"kubernetes.io/projected/5dd0572f-2478-4da0-b0ec-56cb53404123-kube-api-access-g7n5v\") pod \"lws-controller-manager-558b5fdc8c-pv4cf\" (UID: \"5dd0572f-2478-4da0-b0ec-56cb53404123\") " pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:12.098434 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:12.098393 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:12.249921 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:12.249888 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf"] Jul 16 17:24:12.252728 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:24:12.252703 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd0572f_2478_4da0_b0ec_56cb53404123.slice/crio-db61ee07e53b4b97cd6662e434bacbcb25ef7c39cac2fd9ad2f26041192ba477 WatchSource:0}: Error finding container db61ee07e53b4b97cd6662e434bacbcb25ef7c39cac2fd9ad2f26041192ba477: Status 404 returned error can't find the container with id db61ee07e53b4b97cd6662e434bacbcb25ef7c39cac2fd9ad2f26041192ba477 Jul 16 17:24:12.537603 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:12.537524 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:12.620140 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:12.620108 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:12.886168 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:12.886136 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" event={"ID":"5dd0572f-2478-4da0-b0ec-56cb53404123","Type":"ContainerStarted","Data":"db61ee07e53b4b97cd6662e434bacbcb25ef7c39cac2fd9ad2f26041192ba477"} Jul 16 17:24:13.792911 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:13.792881 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:15.900909 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:15.900874 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" event={"ID":"5dd0572f-2478-4da0-b0ec-56cb53404123","Type":"ContainerStarted","Data":"bca304998b955a37f6d99b7ba03809370cf971c365c11c4a70f998a5cb7f9b00"} Jul 16 17:24:15.901380 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:15.900976 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:15.927746 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:15.927701 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" podStartSLOduration=2.040647237 podStartE2EDuration="4.92768438s" totalImagesPullingTime="2.887037143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:24:15.926239994 +0000 UTC m=+578.775746356" watchObservedRunningTime="2026-07-16 17:24:15.92768438 +0000 UTC m=+578.777190781" Jul 16 17:24:16.956856 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:16.956777 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:17.063570 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:17.063534 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:21.373152 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.373117 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j"] Jul 16 17:24:21.376101 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.376076 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.381754 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.381727 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 17:24:21.381933 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.381887 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-djrx4\"" Jul 16 17:24:21.395834 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.395804 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j"] Jul 16 17:24:21.444124 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.444090 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:21.445196 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445175 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-workload-socket\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445214 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-envoy\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445232 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445268 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj2qz\" (UniqueName: \"kubernetes.io/projected/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-kube-api-access-qj2qz\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445389 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-credential-socket\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445389 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445339 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-workload-certs\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445389 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-data\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445389 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445378 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-token\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.445521 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.445398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.515418 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.515385 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:21.546649 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-envoy\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.546855 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546649 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.546855 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546678 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj2qz\" (UniqueName: \"kubernetes.io/projected/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-kube-api-access-qj2qz\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.546855 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546725 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-credential-socket\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.546855 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-workload-certs\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.546855 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546827 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-data\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.547112 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-token\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.547112 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.546988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.547112 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.547067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-workload-socket\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.547257 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.547206 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-workload-certs\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.547314 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.547270 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-data\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.547409 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.547387 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-workload-socket\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.547733 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.547473 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-credential-socket\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.548217 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.548190 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.549716 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.549687 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-envoy\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.549844 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.549780 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.557525 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.557494 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-istio-token\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.557752 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.557718 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj2qz\" (UniqueName: \"kubernetes.io/projected/3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20-kube-api-access-qj2qz\") pod \"openshift-ai-inference-openshift-default-769cdd8d88-mfr9j\" (UID: \"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.690320 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.690240 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:21.835991 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.835960 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j"] Jul 16 17:24:21.838445 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:24:21.838415 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fcc4e1a_8fd5_40c1_98dc_7119ecc39f20.slice/crio-c4c61c751ee7c2a7f2527e784bdf85504e5f4631ed7bfb5f79ee0d54c25ecffb WatchSource:0}: Error finding container c4c61c751ee7c2a7f2527e784bdf85504e5f4631ed7bfb5f79ee0d54c25ecffb: Status 404 returned error can't find the container with id c4c61c751ee7c2a7f2527e784bdf85504e5f4631ed7bfb5f79ee0d54c25ecffb Jul 16 17:24:21.925062 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:21.925026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" event={"ID":"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20","Type":"ContainerStarted","Data":"c4c61c751ee7c2a7f2527e784bdf85504e5f4631ed7bfb5f79ee0d54c25ecffb"} Jul 16 17:24:24.595622 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:24.595582 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 17:24:24.595898 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:24.595659 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 17:24:24.595898 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:24.595688 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 17:24:24.941040 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:24.941006 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" event={"ID":"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20","Type":"ContainerStarted","Data":"2bcd1a8f3b23166419200e15d46f53bc7ec8c9d80fa1edfa0bfd64e00a618816"} Jul 16 17:24:24.965997 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:24.965938 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" podStartSLOduration=1.211146104 podStartE2EDuration="3.965917836s" totalImagesPullingTime="2.754771732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:24:24.965254221 +0000 UTC m=+587.814760581" watchObservedRunningTime="2026-07-16 17:24:24.965917836 +0000 UTC m=+587.815424197" Jul 16 17:24:25.344449 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:25.344414 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:25.691066 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:25.691041 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:25.695412 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:25.695390 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:25.945851 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:25.945748 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:25.946723 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:25.946707 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:24:26.906935 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:26.906906 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-558b5fdc8c-pv4cf" Jul 16 17:24:26.945082 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:26.945050 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:27.036262 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:27.036228 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:30.913995 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:30.913961 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:37.557583 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:37.557559 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:24:37.558043 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:37.557974 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:24:43.464048 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:43.464019 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:47.195557 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:47.195524 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:47.300302 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:47.300271 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:49.261294 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:49.261259 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:51.837303 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:51.837257 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:51.975814 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:51.975757 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:52.525737 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:52.525702 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:54.131982 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:54.131954 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:56.063224 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:56.063191 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:57.105744 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:57.105711 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:57.213760 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:57.213730 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:58.519214 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.519181 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw"] Jul 16 17:24:58.524758 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.524737 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" Jul 16 17:24:58.529303 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.529285 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-sfzpk\"" Jul 16 17:24:58.529662 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.529649 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 17:24:58.530508 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.530493 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 17:24:58.549205 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.549184 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw"] Jul 16 17:24:58.632182 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.632155 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hh2r\" (UniqueName: \"kubernetes.io/projected/14031588-b64f-4567-a3bd-c151b62aa493-kube-api-access-5hh2r\") pod \"limitador-operator-controller-manager-6f9f468797-d7wdw\" (UID: \"14031588-b64f-4567-a3bd-c151b62aa493\") " pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" Jul 16 17:24:58.733003 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.732975 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hh2r\" (UniqueName: \"kubernetes.io/projected/14031588-b64f-4567-a3bd-c151b62aa493-kube-api-access-5hh2r\") pod \"limitador-operator-controller-manager-6f9f468797-d7wdw\" (UID: \"14031588-b64f-4567-a3bd-c151b62aa493\") " pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" Jul 16 17:24:58.793136 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.793061 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hh2r\" (UniqueName: \"kubernetes.io/projected/14031588-b64f-4567-a3bd-c151b62aa493-kube-api-access-5hh2r\") pod \"limitador-operator-controller-manager-6f9f468797-d7wdw\" (UID: \"14031588-b64f-4567-a3bd-c151b62aa493\") " pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" Jul 16 17:24:58.834081 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.834054 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" Jul 16 17:24:58.976763 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:58.976739 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw"] Jul 16 17:24:58.978967 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:24:58.978940 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14031588_b64f_4567_a3bd_c151b62aa493.slice/crio-de2bf190d6f9df3ce329fa71c524cb239f68bfca4cfaab27b43b5746591c65d1 WatchSource:0}: Error finding container de2bf190d6f9df3ce329fa71c524cb239f68bfca4cfaab27b43b5746591c65d1: Status 404 returned error can't find the container with id de2bf190d6f9df3ce329fa71c524cb239f68bfca4cfaab27b43b5746591c65d1 Jul 16 17:24:59.073624 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:59.073544 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" event={"ID":"14031588-b64f-4567-a3bd-c151b62aa493","Type":"ContainerStarted","Data":"de2bf190d6f9df3ce329fa71c524cb239f68bfca4cfaab27b43b5746591c65d1"} Jul 16 17:24:59.160425 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:59.160395 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:59.447096 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:59.447066 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:59.633725 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:59.633694 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:24:59.944010 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:24:59.943980 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:00.046109 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:00.045961 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:03.091546 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:03.091508 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" event={"ID":"14031588-b64f-4567-a3bd-c151b62aa493","Type":"ContainerStarted","Data":"40d80f2e35f13cbe1ed0c0dd801b51d9e5947bac415bbe5aff8a15c17d073bbf"} Jul 16 17:25:03.091985 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:03.091648 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" Jul 16 17:25:03.127047 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:03.127003 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" podStartSLOduration=1.958356743 podStartE2EDuration="5.126989986s" totalImagesPullingTime="3.168633243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:24:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:25:03.126631339 +0000 UTC m=+625.976137699" watchObservedRunningTime="2026-07-16 17:25:03.126989986 +0000 UTC m=+625.976496346" Jul 16 17:25:04.654502 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:04.654470 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:04.723299 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:04.723272 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:04.866100 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:04.866071 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:04.934215 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:04.934139 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:06.009621 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:06.009592 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:06.081730 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:06.081707 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:13.975808 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:13.975757 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:14.054773 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:14.054739 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:14.098408 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:14.098381 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw" Jul 16 17:25:14.602840 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:14.602808 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:14.712753 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:14.712720 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:18.383736 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:18.383704 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:18.487644 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:18.487615 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:22.577885 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:22.577831 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:22.688417 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:22.688384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:27.772204 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:27.772166 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:27.876627 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:27.876594 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:28.547851 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:28.547816 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:28.655715 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:28.655682 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:29.018158 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:29.018128 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:29.104300 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:29.104272 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:38.304270 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:38.304237 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:38.412481 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:38.412457 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:39.942460 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:39.942428 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:40.041424 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.041390 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:40.127132 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.127105 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:40.437919 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.437886 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-nzkh5"] Jul 16 17:25:40.445749 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.445724 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.448464 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.448430 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 17:25:40.448547 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.448529 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cppvr\"" Jul 16 17:25:40.455986 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.455968 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-nzkh5"] Jul 16 17:25:40.506862 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.506841 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-nzkh5"] Jul 16 17:25:40.577873 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.577847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb-config-file\") pod \"limitador-limitador-69574b596d-nzkh5\" (UID: \"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb\") " pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.577968 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.577887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxf2m\" (UniqueName: \"kubernetes.io/projected/6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb-kube-api-access-xxf2m\") pod \"limitador-limitador-69574b596d-nzkh5\" (UID: \"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb\") " pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.679019 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.678988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb-config-file\") pod \"limitador-limitador-69574b596d-nzkh5\" (UID: \"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb\") " pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.679166 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.679027 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxf2m\" (UniqueName: \"kubernetes.io/projected/6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb-kube-api-access-xxf2m\") pod \"limitador-limitador-69574b596d-nzkh5\" (UID: \"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb\") " pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.679646 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.679621 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb-config-file\") pod \"limitador-limitador-69574b596d-nzkh5\" (UID: \"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb\") " pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.706694 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.706639 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxf2m\" (UniqueName: \"kubernetes.io/projected/6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb-kube-api-access-xxf2m\") pod \"limitador-limitador-69574b596d-nzkh5\" (UID: \"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb\") " pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.755435 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.755412 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:40.882050 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:40.882024 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-69574b596d-nzkh5"] Jul 16 17:25:40.883773 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:25:40.883748 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c9c1bde_6da0_4e69_b3db_b7cf39c0d3fb.slice/crio-1eec2a40cd893f35d3892663a4dbdc305c41257317937cb3a2d557fc3f31f991 WatchSource:0}: Error finding container 1eec2a40cd893f35d3892663a4dbdc305c41257317937cb3a2d557fc3f31f991: Status 404 returned error can't find the container with id 1eec2a40cd893f35d3892663a4dbdc305c41257317937cb3a2d557fc3f31f991 Jul 16 17:25:41.029610 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:41.029547 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:41.121692 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:41.121664 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:41.226715 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:41.226692 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:41.242239 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:41.242212 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" event={"ID":"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb","Type":"ContainerStarted","Data":"1eec2a40cd893f35d3892663a4dbdc305c41257317937cb3a2d557fc3f31f991"} Jul 16 17:25:41.322280 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:41.322184 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:43.811410 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:43.811380 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:43.941506 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:43.941451 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:45.262206 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:45.262131 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" event={"ID":"6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb","Type":"ContainerStarted","Data":"1f5aaf4dc86d733b2d142ea414e6ce3cdbc44f71278c67ace190ae59858d6bc8"} Jul 16 17:25:45.262526 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:45.262276 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:45.294444 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:45.294396 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" podStartSLOduration=1.263250256 podStartE2EDuration="5.294383534s" totalImagesPullingTime="4.031133278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:25:45.293383935 +0000 UTC m=+668.142890295" watchObservedRunningTime="2026-07-16 17:25:45.294383534 +0000 UTC m=+668.143889893" Jul 16 17:25:45.938381 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:45.938352 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:46.036070 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:46.036040 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:49.226098 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:49.226064 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:49.290762 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:49.290734 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:49.608999 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:49.608907 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:49.703155 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:49.703115 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:50.000432 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:50.000403 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:50.065068 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:50.065035 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:56.267023 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:56.266991 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-69574b596d-nzkh5" Jul 16 17:25:58.092221 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:58.092182 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:25:58.206151 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:25:58.206117 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:01.015151 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:01.015117 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:02.749399 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:02.749358 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:02.957924 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:02.957896 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:03.322292 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:03.322257 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:21.321665 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:21.321621 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:21.387769 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:21.387717 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:28.738526 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:28.738487 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:43.073697 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:43.073635 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:47.768334 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:47.768301 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:47.883493 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:47.883451 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:48.168861 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:48.168812 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:48.262905 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:48.262863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:51.040611 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:51.040571 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:26:51.141585 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:26:51.141557 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:27:14.653126 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:27:14.653064 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:27:16.264445 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:27:16.264399 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:27:24.434147 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:27:24.434111 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:28:23.356669 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:23.356594 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:28:23.465728 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:23.465697 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:28:47.547369 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:47.547336 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:28:47.919355 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:47.919328 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:28:56.986962 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:56.986915 2577 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 17:28:56.987347 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:56.987003 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" podUID="3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 17:28:58.024336 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:58.024311 2577 generic.go:356] "Generic (PLEG): container finished" podID="3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20" containerID="2bcd1a8f3b23166419200e15d46f53bc7ec8c9d80fa1edfa0bfd64e00a618816" exitCode=0 Jul 16 17:28:58.024704 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:58.024340 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" event={"ID":"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20","Type":"ContainerDied","Data":"2bcd1a8f3b23166419200e15d46f53bc7ec8c9d80fa1edfa0bfd64e00a618816"} Jul 16 17:28:58.024704 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:58.024643 2577 scope.go:117] "RemoveContainer" containerID="2bcd1a8f3b23166419200e15d46f53bc7ec8c9d80fa1edfa0bfd64e00a618816" Jul 16 17:28:58.025514 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:58.025488 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 17:28:58.025565 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:58.025556 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 17:28:58.025597 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:58.025585 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 17:28:59.032872 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:59.032821 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" event={"ID":"3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20","Type":"ContainerStarted","Data":"54a8656fd6e1c695d1c5566d839b4350706f58afa260e0cbb66ea9ef17009836"} Jul 16 17:28:59.691322 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:59.691292 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:28:59.691476 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:59.691448 2577 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 17:28:59.691519 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:28:59.691490 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" podUID="3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 17:29:00.691358 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:00.691319 2577 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 17:29:00.691837 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:00.691373 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" podUID="3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 17:29:01.321086 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.321053 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx"] Jul 16 17:29:01.328811 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.328028 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.331693 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.331548 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 16 17:29:01.331825 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.331780 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 16 17:29:01.334726 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.334701 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"fc-smoke-test-kserve-self-signed-certs\"" Jul 16 17:29:01.337265 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.335311 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-sdm8r\"" Jul 16 17:29:01.339501 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.339478 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx"] Jul 16 17:29:01.390583 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.390554 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:29:01.448992 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.448963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-dshm\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.449135 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.449004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-tmp-dir\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.449135 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.449033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-model-cache\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.449135 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.449049 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vljlj\" (UniqueName: \"kubernetes.io/projected/1e491ead-ea2e-47bf-ada9-3cc83480e879-kube-api-access-vljlj\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.449135 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.449068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e491ead-ea2e-47bf-ada9-3cc83480e879-tls-certs\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.449135 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.449103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-home\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.549915 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.549868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-dshm\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.549915 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.549927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-tmp-dir\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.550181 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.549959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-model-cache\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.550181 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.549974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljlj\" (UniqueName: \"kubernetes.io/projected/1e491ead-ea2e-47bf-ada9-3cc83480e879-kube-api-access-vljlj\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.550181 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.549994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e491ead-ea2e-47bf-ada9-3cc83480e879-tls-certs\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.550181 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.550022 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-home\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.550454 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.550430 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-model-cache\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.550589 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.550485 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-home\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.550656 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.550601 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-tmp-dir\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.552153 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.552134 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e491ead-ea2e-47bf-ada9-3cc83480e879-dshm\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.552306 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.552287 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e491ead-ea2e-47bf-ada9-3cc83480e879-tls-certs\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.561301 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.561281 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljlj\" (UniqueName: \"kubernetes.io/projected/1e491ead-ea2e-47bf-ada9-3cc83480e879-kube-api-access-vljlj\") pod \"fc-smoke-test-kserve-7f4687897-r5tjx\" (UID: \"1e491ead-ea2e-47bf-ada9-3cc83480e879\") " pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.651099 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.651072 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:01.687368 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.687342 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:29:01.690639 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.690615 2577 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 17:29:01.690739 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.690689 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" podUID="3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 17:29:01.810782 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.810751 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx"] Jul 16 17:29:01.813925 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:29:01.813899 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e491ead_ea2e_47bf_ada9_3cc83480e879.slice/crio-f9bbe14e0812d0954c60abb5712d19ca71f782636a811cde30b6caaacc261604 WatchSource:0}: Error finding container f9bbe14e0812d0954c60abb5712d19ca71f782636a811cde30b6caaacc261604: Status 404 returned error can't find the container with id f9bbe14e0812d0954c60abb5712d19ca71f782636a811cde30b6caaacc261604 Jul 16 17:29:01.815673 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:01.815658 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 17:29:02.052690 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:02.052611 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" event={"ID":"1e491ead-ea2e-47bf-ada9-3cc83480e879","Type":"ContainerStarted","Data":"f9bbe14e0812d0954c60abb5712d19ca71f782636a811cde30b6caaacc261604"} Jul 16 17:29:02.691223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:02.691189 2577 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 17:29:02.691403 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:02.691246 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" podUID="3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 17:29:03.690976 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:03.690895 2577 patch_prober.go:28] interesting pod/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 16 17:29:03.690976 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:03.690945 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" podUID="3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 16 17:29:04.064854 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:04.064755 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" event={"ID":"1e491ead-ea2e-47bf-ada9-3cc83480e879","Type":"ContainerStarted","Data":"0a1a0ee052d6290825fab633da1694113b3ca777a2c21b89c3d874fadf27188b"} Jul 16 17:29:04.095171 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:04.095117 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" podStartSLOduration=1.49454388 podStartE2EDuration="3.095100579s" totalImagesPullingTime="1.600556699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:29:04.093821343 +0000 UTC m=+866.943327704" watchObservedRunningTime="2026-07-16 17:29:04.095100579 +0000 UTC m=+866.944606970" Jul 16 17:29:04.653549 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:04.653518 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:04.693734 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:04.693715 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:29:04.694215 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:04.694201 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:29:04.694896 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:04.694876 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j" Jul 16 17:29:11.653129 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:11.653092 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:11.664965 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:11.664938 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:12.106903 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:12.106827 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx" Jul 16 17:29:18.199770 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:18.199728 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:29:25.796695 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:25.796663 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:29:37.590207 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:37.590133 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:29:37.591608 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:29:37.591583 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:30:07.326426 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:30:07.326367 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:30:29.356049 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:30:29.356017 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:30:29.857360 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:30:29.857335 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:30:53.586166 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:30:53.586135 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:30:54.045882 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:30:54.045830 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:31:05.289009 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:31:05.288927 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:31:30.689239 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:31:30.689203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:31:30.812886 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:31:30.812855 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:31:53.393980 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:31:53.393939 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:31:53.834942 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:31:53.834852 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:32:11.570265 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:32:11.570232 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:33:06.508376 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:33:06.508338 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:33:06.582449 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:33:06.582408 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:33:16.194969 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:33:16.194931 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:33:40.012880 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:33:40.012839 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:33:40.086780 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:33:40.086745 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:33:53.256294 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:33:53.256262 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:10.057897 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:34:10.057820 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:10.130087 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:34:10.130059 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:16.855182 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:34:16.855152 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:37.618851 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:34:37.618778 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:34:37.619859 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:34:37.619834 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:34:43.952598 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:34:43.952569 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:34:44.076991 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:34:44.076956 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:11.857223 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:35:11.857188 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:12.082852 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:35:12.082786 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:12.183048 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:35:12.182981 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:35.498641 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:35:35.498572 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:35:35.598347 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:35:35.598317 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:38:11.397834 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:38:11.397801 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:39:37.655257 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:39:37.655230 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:39:37.658147 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:39:37.658128 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:40:30.042537 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:40:30.042459 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:44:22.717055 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:44:22.717024 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:44:24.073662 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:44:24.073630 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:44:37.688257 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:44:37.688141 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:44:37.688257 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:44:37.688230 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:44:45.565507 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:44:45.565403 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:44:45.834545 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:44:45.834473 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:44:52.326913 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:44:52.326873 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:45:29.466232 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:45:29.466198 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:48:37.068815 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:48:37.068759 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:49:37.715505 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:49:37.715407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:49:37.719535 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:49:37.718015 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:51:13.429191 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.429153 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs"] Jul 16 17:51:13.432619 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.432546 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.435579 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.435551 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\"" Jul 16 17:51:13.451813 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.451767 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs"] Jul 16 17:51:13.486666 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.486625 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:51:13.554671 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.554621 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96495a17-94db-46ed-8b38-53c7c97e598e-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.554671 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.554668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.554904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.554739 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.554904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.554756 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg2hh\" (UniqueName: \"kubernetes.io/projected/96495a17-94db-46ed-8b38-53c7c97e598e-kube-api-access-mg2hh\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.554904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.554818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.554904 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.554872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.655444 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96495a17-94db-46ed-8b38-53c7c97e598e-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.655444 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.655718 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655480 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.655718 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg2hh\" (UniqueName: \"kubernetes.io/projected/96495a17-94db-46ed-8b38-53c7c97e598e-kube-api-access-mg2hh\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.655718 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.655718 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655551 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.655975 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655882 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.656024 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.655982 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.656103 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.656078 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.657780 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.657758 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96495a17-94db-46ed-8b38-53c7c97e598e-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.658069 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.658052 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96495a17-94db-46ed-8b38-53c7c97e598e-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.665980 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.665945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg2hh\" (UniqueName: \"kubernetes.io/projected/96495a17-94db-46ed-8b38-53c7c97e598e-kube-api-access-mg2hh\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs\" (UID: \"96495a17-94db-46ed-8b38-53c7c97e598e\") " pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.742775 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.742680 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:13.882073 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:13.882049 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs"] Jul 16 17:51:13.884963 ip-10-0-133-160 kubenswrapper[2577]: W0716 17:51:13.884933 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96495a17_94db_46ed_8b38_53c7c97e598e.slice/crio-8b54afdd26aa0d2851cccb27d588e0f14f3e387edb2c57040de47f1f6d9a5475 WatchSource:0}: Error finding container 8b54afdd26aa0d2851cccb27d588e0f14f3e387edb2c57040de47f1f6d9a5475: Status 404 returned error can't find the container with id 8b54afdd26aa0d2851cccb27d588e0f14f3e387edb2c57040de47f1f6d9a5475 Jul 16 17:51:14.793254 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:14.793216 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" event={"ID":"96495a17-94db-46ed-8b38-53c7c97e598e","Type":"ContainerStarted","Data":"81d0a7e4bf4060017f31b266e9c26714d6d1d4890c77fdc94bca19ae516d049a"} Jul 16 17:51:14.793649 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:14.793269 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" event={"ID":"96495a17-94db-46ed-8b38-53c7c97e598e","Type":"ContainerStarted","Data":"8b54afdd26aa0d2851cccb27d588e0f14f3e387edb2c57040de47f1f6d9a5475"} Jul 16 17:51:14.813691 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:14.813633 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" podStartSLOduration=1.813618135 podStartE2EDuration="1.813618135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 17:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 17:51:14.812185651 +0000 UTC m=+2197.661692011" watchObservedRunningTime="2026-07-16 17:51:14.813618135 +0000 UTC m=+2197.663124494" Jul 16 17:51:15.088129 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:15.088041 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:51:15.743111 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:15.743071 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:23.743395 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:23.743305 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:23.755544 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:23.755519 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:23.844020 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:23.843990 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs" Jul 16 17:51:40.129359 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:40.129314 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:51:41.903022 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:51:41.902972 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:53:13.806530 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:53:13.806490 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:54:37.739708 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:54:37.739607 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:54:37.744538 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:54:37.744518 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:55:23.269361 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:55:23.269280 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:57:13.978085 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:57:13.978035 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 17:59:37.765176 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:59:37.765074 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 17:59:37.772258 ip-10-0-133-160 kubenswrapper[2577]: I0716 17:59:37.772239 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:00:21.091558 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.091525 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l"] Jul 16 18:00:21.095491 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.095470 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.098162 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.098142 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-1-openshift-default-dockercfg-8p2bv\"" Jul 16 18:00:21.098276 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.098148 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\"" Jul 16 18:00:21.118337 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.118313 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l"] Jul 16 18:00:21.136449 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.136424 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:00:21.190303 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190278 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190421 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190421 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190421 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbwfs\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-kube-api-access-hbwfs\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190513 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190732 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.190732 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.190637 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.199116 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.199087 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:00:21.291623 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.291730 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291633 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.291730 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291660 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.291730 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.291730 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbwfs\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-kube-api-access-hbwfs\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292035 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292035 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292035 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292035 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.291895 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292250 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.292050 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292250 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.292096 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-certs\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292542 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.292517 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-credential-socket\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292630 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.292576 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-data\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.292849 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.292828 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.294121 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.294103 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-envoy\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.294260 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.294243 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-podinfo\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.301760 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.301741 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbwfs\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-kube-api-access-hbwfs\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.301943 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.301925 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-token\") pod \"router-gateway-1-openshift-default-799f46c59b-gx98l\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.407473 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.407450 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:21.540223 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.540200 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l"] Jul 16 18:00:21.542424 ip-10-0-133-160 kubenswrapper[2577]: W0716 18:00:21.542396 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96aead4a_1ea0_449a_a22a_e7bd30fbcfa9.slice/crio-e35cf4d3dd2e0a60988411cf1b646fe94d08e9d83e5c50199becfea82f521130 WatchSource:0}: Error finding container e35cf4d3dd2e0a60988411cf1b646fe94d08e9d83e5c50199becfea82f521130: Status 404 returned error can't find the container with id e35cf4d3dd2e0a60988411cf1b646fe94d08e9d83e5c50199becfea82f521130 Jul 16 18:00:21.544466 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.544437 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:00:21.544553 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.544498 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:00:21.544553 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:21.544528 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:00:22.157180 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:22.157146 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" event={"ID":"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9","Type":"ContainerStarted","Data":"6c25a4d412986d6248f0f71e2e3c2df82637e9def120f14068f9313a4dbc7057"} Jul 16 18:00:22.157555 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:22.157204 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" event={"ID":"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9","Type":"ContainerStarted","Data":"e35cf4d3dd2e0a60988411cf1b646fe94d08e9d83e5c50199becfea82f521130"} Jul 16 18:00:22.183980 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:22.183937 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podStartSLOduration=1.183922218 podStartE2EDuration="1.183922218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 18:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:00:22.182211239 +0000 UTC m=+2745.031717614" watchObservedRunningTime="2026-07-16 18:00:22.183922218 +0000 UTC m=+2745.033428579" Jul 16 18:00:22.407646 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:22.407565 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:22.423100 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:22.423074 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:23.161518 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:23.161486 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:23.162312 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:23.162295 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:24.601648 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:24.601619 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:00:29.024585 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:29.024553 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:00:35.005579 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.005546 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr"] Jul 16 18:00:35.009383 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.009365 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.023635 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.023601 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr"] Jul 16 18:00:35.116543 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.116732 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116554 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-credential-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.116732 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116582 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-podinfo\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.116732 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-certs\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.116732 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.116732 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116716 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-envoy\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.117020 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116761 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-data\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.117020 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116816 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n29r\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-kube-api-access-6n29r\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.117020 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.116846 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-token\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.118350 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.118326 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr"] Jul 16 18:00:35.118655 ip-10-0-133-160 kubenswrapper[2577]: E0716 18:00:35.118628 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[credential-socket istio-data istio-envoy istio-podinfo istio-token istiod-ca-cert kube-api-access-6n29r workload-certs workload-socket], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" podUID="4fbbf3b1-50b4-4f11-962f-24a0890ad097" Jul 16 18:00:35.213800 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.213770 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.217726 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.217703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.217863 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.217743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-credential-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.217863 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.217775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-podinfo\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.217978 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.217876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-certs\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.217978 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.217913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.217978 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.217948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-envoy\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218132 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218000 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-data\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218132 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n29r\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-kube-api-access-6n29r\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218132 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218056 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-token\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218286 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218188 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-credential-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218286 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218221 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218407 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218382 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-certs\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218471 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218411 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-data\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.218747 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.218716 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.220323 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.220300 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-envoy\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.220697 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.220674 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-podinfo\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.220777 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.220723 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.228888 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.228833 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n29r\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-kube-api-access-6n29r\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.233536 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.229368 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-token\") pod \"router-gateway-1-openshift-default-54c47cdfc7-qbmpr\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:35.319001 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.318934 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-credential-socket\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319130 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319005 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-podinfo\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319130 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319033 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-token\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319130 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319067 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-data\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319130 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319097 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-socket\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319316 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319147 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-envoy\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319316 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319190 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-certs\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319316 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319190 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:00:35.319316 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319218 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istiod-ca-cert\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319316 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319241 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n29r\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-kube-api-access-6n29r\") pod \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\" (UID: \"4fbbf3b1-50b4-4f11-962f-24a0890ad097\") " Jul 16 18:00:35.319569 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319342 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-data" (OuterVolumeSpecName: "istio-data") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:00:35.319569 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319391 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:00:35.319688 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319583 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-credential-socket\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.319688 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319601 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-data\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.319688 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319615 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-socket\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.319688 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319632 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 18:00:35.319925 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.319741 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:00:35.321200 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.321174 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 18:00:35.321311 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.321289 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-kube-api-access-6n29r" (OuterVolumeSpecName: "kube-api-access-6n29r") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "kube-api-access-6n29r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 18:00:35.321585 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.321559 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:00:35.321677 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.321563 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-token" (OuterVolumeSpecName: "istio-token") pod "4fbbf3b1-50b4-4f11-962f-24a0890ad097" (UID: "4fbbf3b1-50b4-4f11-962f-24a0890ad097"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 18:00:35.420634 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.420591 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-podinfo\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.420784 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.420638 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-token\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.420784 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.420659 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istio-envoy\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.420784 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.420679 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fbbf3b1-50b4-4f11-962f-24a0890ad097-workload-certs\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.420784 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.420698 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fbbf3b1-50b4-4f11-962f-24a0890ad097-istiod-ca-cert\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:35.420784 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:35.420717 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n29r\" (UniqueName: \"kubernetes.io/projected/4fbbf3b1-50b4-4f11-962f-24a0890ad097-kube-api-access-6n29r\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:00:36.228602 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:36.228560 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr" Jul 16 18:00:36.276662 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:36.276631 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr"] Jul 16 18:00:36.282062 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:36.282039 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr"] Jul 16 18:00:37.687339 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:37.687306 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fbbf3b1-50b4-4f11-962f-24a0890ad097" path="/var/lib/kubelet/pods/4fbbf3b1-50b4-4f11-962f-24a0890ad097/volumes" Jul 16 18:00:39.203945 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:39.203905 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 18:00:41.255394 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:41.255360 2577 generic.go:356] "Generic (PLEG): container finished" podID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerID="6c25a4d412986d6248f0f71e2e3c2df82637e9def120f14068f9313a4dbc7057" exitCode=0 Jul 16 18:00:41.255834 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:41.255431 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" event={"ID":"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9","Type":"ContainerDied","Data":"6c25a4d412986d6248f0f71e2e3c2df82637e9def120f14068f9313a4dbc7057"} Jul 16 18:00:41.255834 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:41.255827 2577 scope.go:117] "RemoveContainer" containerID="6c25a4d412986d6248f0f71e2e3c2df82637e9def120f14068f9313a4dbc7057" Jul 16 18:00:41.256903 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:41.256880 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:00:41.256978 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:41.256944 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:00:41.256978 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:41.256973 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:00:42.263072 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:42.263032 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" event={"ID":"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9","Type":"ContainerStarted","Data":"533a42f7bdc7889d69ec389502a3a1ef1a10f690daca50363a485db168849f70"} Jul 16 18:00:42.407758 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:42.407731 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:42.407944 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:42.407911 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 18:00:43.408336 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:43.408297 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 18:00:44.408703 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:44.408658 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 18:00:45.201813 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:45.201763 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:00:45.408244 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:45.408204 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 18:00:46.408739 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:46.408699 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 16 18:00:47.726584 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:47.726545 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:47.727120 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:47.727004 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:00:47.727837 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:00:47.727813 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:01:05.655143 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:05.655048 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:01:09.833517 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.833477 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn"] Jul 16 18:01:09.836138 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.836117 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.853744 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.853716 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn"] Jul 16 18:01:09.937146 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937108 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-workload-certs\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937146 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-workload-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937362 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-podinfo\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937362 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1686a54a-6448-42ed-ac6e-9c9356d279d4-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937362 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-credential-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937362 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-data\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937362 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-envoy\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937562 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937375 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgqbz\" (UniqueName: \"kubernetes.io/projected/1686a54a-6448-42ed-ac6e-9c9356d279d4-kube-api-access-hgqbz\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:09.937562 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:09.937415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-token\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038369 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038333 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1686a54a-6448-42ed-ac6e-9c9356d279d4-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038369 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038371 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-credential-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038392 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-data\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038425 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-envoy\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgqbz\" (UniqueName: \"kubernetes.io/projected/1686a54a-6448-42ed-ac6e-9c9356d279d4-kube-api-access-hgqbz\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-token\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038766 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038594 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-workload-certs\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038766 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-workload-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038766 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-podinfo\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038947 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038853 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-credential-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.038999 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.038971 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-workload-socket\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.039108 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.039082 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-data\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.039362 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.039340 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-workload-certs\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.039569 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.039543 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1686a54a-6448-42ed-ac6e-9c9356d279d4-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.040687 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.040667 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-envoy\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.041391 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.041368 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-podinfo\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.049447 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.049428 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1686a54a-6448-42ed-ac6e-9c9356d279d4-istio-token\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.049541 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.049529 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgqbz\" (UniqueName: \"kubernetes.io/projected/1686a54a-6448-42ed-ac6e-9c9356d279d4-kube-api-access-hgqbz\") pod \"router-gateway-1-openshift-default-54c47cdfc7-kfbtn\" (UID: \"1686a54a-6448-42ed-ac6e-9c9356d279d4\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.149346 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.149307 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:10.286345 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.286319 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn"] Jul 16 18:01:10.288336 ip-10-0-133-160 kubenswrapper[2577]: W0716 18:01:10.288295 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1686a54a_6448_42ed_ac6e_9c9356d279d4.slice/crio-2c81ddc9aaf1d94cc5404055fe33fd269edac1a83c11acdc614b99775c43a1fb WatchSource:0}: Error finding container 2c81ddc9aaf1d94cc5404055fe33fd269edac1a83c11acdc614b99775c43a1fb: Status 404 returned error can't find the container with id 2c81ddc9aaf1d94cc5404055fe33fd269edac1a83c11acdc614b99775c43a1fb Jul 16 18:01:10.290506 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.290476 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:01:10.290608 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.290548 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:01:10.290608 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.290592 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 16 18:01:10.395037 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.394997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" event={"ID":"1686a54a-6448-42ed-ac6e-9c9356d279d4","Type":"ContainerStarted","Data":"2d5d950536a48904069cade8d69941c86b26bd8751540e06fea2ef6d08ec5d39"} Jul 16 18:01:10.395190 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.395057 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" event={"ID":"1686a54a-6448-42ed-ac6e-9c9356d279d4","Type":"ContainerStarted","Data":"2c81ddc9aaf1d94cc5404055fe33fd269edac1a83c11acdc614b99775c43a1fb"} Jul 16 18:01:10.418733 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:10.418615 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" podStartSLOduration=1.418599416 podStartE2EDuration="1.418599416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 18:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:01:10.417644638 +0000 UTC m=+2793.267150999" watchObservedRunningTime="2026-07-16 18:01:10.418599416 +0000 UTC m=+2793.268105776" Jul 16 18:01:11.149979 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:11.149934 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:11.151441 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:11.151407 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" podUID="1686a54a-6448-42ed-ac6e-9c9356d279d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 18:01:12.150407 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:12.150354 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" podUID="1686a54a-6448-42ed-ac6e-9c9356d279d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 18:01:13.149722 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:13.149676 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" podUID="1686a54a-6448-42ed-ac6e-9c9356d279d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 18:01:14.150721 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:14.150667 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" podUID="1686a54a-6448-42ed-ac6e-9c9356d279d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 18:01:15.150288 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:15.150241 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" podUID="1686a54a-6448-42ed-ac6e-9c9356d279d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 18:01:16.150752 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:16.150695 2577 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" podUID="1686a54a-6448-42ed-ac6e-9c9356d279d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 16 18:01:16.476089 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:16.475987 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:01:17.154733 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:17.154696 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:17.155207 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:17.155121 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:17.156266 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:17.156240 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn" Jul 16 18:01:17.256444 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:17.254093 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l"] Jul 16 18:01:17.256444 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:17.254410 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" containerID="cri-o://533a42f7bdc7889d69ec389502a3a1ef1a10f690daca50363a485db168849f70" gracePeriod=30 Jul 16 18:01:17.729287 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:17.729236 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 18:01:22.451897 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.451860 2577 generic.go:356] "Generic (PLEG): container finished" podID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerID="533a42f7bdc7889d69ec389502a3a1ef1a10f690daca50363a485db168849f70" exitCode=0 Jul 16 18:01:22.452264 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.451937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" event={"ID":"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9","Type":"ContainerDied","Data":"533a42f7bdc7889d69ec389502a3a1ef1a10f690daca50363a485db168849f70"} Jul 16 18:01:22.452264 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.452015 2577 scope.go:117] "RemoveContainer" containerID="6c25a4d412986d6248f0f71e2e3c2df82637e9def120f14068f9313a4dbc7057" Jul 16 18:01:22.502745 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.502726 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:01:22.557836 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.557753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-envoy\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.557836 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.557799 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-podinfo\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.557836 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.557825 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-data\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.558100 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.557844 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-credential-socket\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.558100 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.557885 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbwfs\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-kube-api-access-hbwfs\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.558100 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.557960 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-certs\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.558100 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.557983 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-socket\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.558100 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558005 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istiod-ca-cert\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.558100 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558034 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-token\") pod \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\" (UID: \"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9\") " Jul 16 18:01:22.558402 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558161 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-data" (OuterVolumeSpecName: "istio-data") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:01:22.558402 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558327 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:01:22.558402 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558351 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-data\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.558402 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558353 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:01:22.558610 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558571 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:01:22.558750 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.558729 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 18:01:22.560163 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.560141 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-kube-api-access-hbwfs" (OuterVolumeSpecName: "kube-api-access-hbwfs") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "kube-api-access-hbwfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 18:01:22.560267 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.560169 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 18:01:22.560267 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.560174 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:01:22.560267 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.560223 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-token" (OuterVolumeSpecName: "istio-token") pod "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" (UID: "96aead4a-1ea0-449a-a22a-e7bd30fbcfa9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 18:01:22.659124 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659091 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-envoy\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.659124 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659119 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-podinfo\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.659124 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659128 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-credential-socket\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.659356 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659136 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbwfs\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-kube-api-access-hbwfs\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.659356 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659146 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-certs\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.659356 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659154 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-workload-socket\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.659356 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659163 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istiod-ca-cert\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:22.659356 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:22.659171 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9-istio-token\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:01:23.458076 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:23.458041 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" event={"ID":"96aead4a-1ea0-449a-a22a-e7bd30fbcfa9","Type":"ContainerDied","Data":"e35cf4d3dd2e0a60988411cf1b646fe94d08e9d83e5c50199becfea82f521130"} Jul 16 18:01:23.458076 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:23.458060 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l" Jul 16 18:01:23.458552 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:23.458100 2577 scope.go:117] "RemoveContainer" containerID="533a42f7bdc7889d69ec389502a3a1ef1a10f690daca50363a485db168849f70" Jul 16 18:01:23.493698 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:23.493668 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l"] Jul 16 18:01:23.495624 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:23.495600 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l"] Jul 16 18:01:23.612219 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:23.612187 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:01:23.688671 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:01:23.688634 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" path="/var/lib/kubelet/pods/96aead4a-1ea0-449a-a22a-e7bd30fbcfa9/volumes" Jul 16 18:02:32.686942 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:02:32.686906 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:04:07.439197 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:04:07.439116 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:04:07.516594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:04:07.516562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:04:22.493638 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:04:22.493603 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:04:37.791603 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:04:37.791578 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:04:37.798686 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:04:37.798665 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:06:52.056588 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:06:52.056554 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:07:16.097986 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:07:16.097892 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:16.339563 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:07:16.339522 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:07:47.992709 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:07:47.992672 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:09:32.012352 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:09:32.012315 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:09:32.072768 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:09:32.072734 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:09:37.817819 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:09:37.817777 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:09:37.826568 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:09:37.826544 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:09:55.915528 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:09:55.915486 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:09:56.222851 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:09:56.222749 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:10:28.514272 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:10:28.514234 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:10:32.692303 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:10:32.692265 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:10:47.833163 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:10:47.833130 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:10:47.934386 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:10:47.934348 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:13:39.852740 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:13:39.852664 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:14:37.844305 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:14:37.844277 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:14:37.853910 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:14:37.853891 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:15:47.038148 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:15:47.038119 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:16:11.451343 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:16:11.451250 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:19:37.872329 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:19:37.872301 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:19:37.882393 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:19:37.882373 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:20:39.726898 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:20:39.726786 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:20:40.756822 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:20:40.756779 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:20:50.310736 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:20:50.310700 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:21:54.530502 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:21:54.530468 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:16.438998 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:24:16.438958 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:24:37.903974 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:24:37.903946 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:24:37.912015 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:24:37.911997 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wsjz_3c274373-f9bb-42eb-bc62-4b247a4df6f0/ovn-acl-logging/0.log" Jul 16 18:26:24.587817 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.587764 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j9rsm/must-gather-w7m6q"] Jul 16 18:26:24.588238 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.588136 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" Jul 16 18:26:24.588238 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.588148 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" Jul 16 18:26:24.588361 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.588257 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" Jul 16 18:26:24.588361 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.588298 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" Jul 16 18:26:24.588454 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.588391 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" Jul 16 18:26:24.588454 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.588404 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96aead4a-1ea0-449a-a22a-e7bd30fbcfa9" containerName="istio-proxy" Jul 16 18:26:24.591657 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.591640 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:24.594730 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.594706 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j9rsm\"/\"openshift-service-ca.crt\"" Jul 16 18:26:24.594856 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.594747 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j9rsm\"/\"kube-root-ca.crt\"" Jul 16 18:26:24.612450 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.612424 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j9rsm/must-gather-w7m6q"] Jul 16 18:26:24.636905 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.636866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:26:24.717410 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.717370 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47c9v\" (UniqueName: \"kubernetes.io/projected/1077b595-1192-44e0-89fc-8044e905eda5-kube-api-access-47c9v\") pod \"must-gather-w7m6q\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:24.717410 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.717415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1077b595-1192-44e0-89fc-8044e905eda5-must-gather-output\") pod \"must-gather-w7m6q\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:24.818870 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.818828 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47c9v\" (UniqueName: \"kubernetes.io/projected/1077b595-1192-44e0-89fc-8044e905eda5-kube-api-access-47c9v\") pod \"must-gather-w7m6q\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:24.819058 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.818881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1077b595-1192-44e0-89fc-8044e905eda5-must-gather-output\") pod \"must-gather-w7m6q\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:24.819250 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.819226 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1077b595-1192-44e0-89fc-8044e905eda5-must-gather-output\") pod \"must-gather-w7m6q\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:24.833724 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.833693 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47c9v\" (UniqueName: \"kubernetes.io/projected/1077b595-1192-44e0-89fc-8044e905eda5-kube-api-access-47c9v\") pod \"must-gather-w7m6q\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:24.900871 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:24.900831 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:26:25.037099 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:25.037074 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j9rsm/must-gather-w7m6q"] Jul 16 18:26:25.039664 ip-10-0-133-160 kubenswrapper[2577]: W0716 18:26:25.039637 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1077b595_1192_44e0_89fc_8044e905eda5.slice/crio-97b5b737d425b1f39b6a3a367572d0f7c24f5c30f70dcd1c638c64f4e6cee2f8 WatchSource:0}: Error finding container 97b5b737d425b1f39b6a3a367572d0f7c24f5c30f70dcd1c638c64f4e6cee2f8: Status 404 returned error can't find the container with id 97b5b737d425b1f39b6a3a367572d0f7c24f5c30f70dcd1c638c64f4e6cee2f8 Jul 16 18:26:25.041399 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:25.041384 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 18:26:25.334669 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:25.334576 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" event={"ID":"1077b595-1192-44e0-89fc-8044e905eda5","Type":"ContainerStarted","Data":"97b5b737d425b1f39b6a3a367572d0f7c24f5c30f70dcd1c638c64f4e6cee2f8"} Jul 16 18:26:26.612011 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:26.611974 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:26:32.380164 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:32.380122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" event={"ID":"1077b595-1192-44e0-89fc-8044e905eda5","Type":"ContainerStarted","Data":"7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9"} Jul 16 18:26:32.380557 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:32.380176 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" event={"ID":"1077b595-1192-44e0-89fc-8044e905eda5","Type":"ContainerStarted","Data":"f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373"} Jul 16 18:26:32.399214 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:32.399161 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" podStartSLOduration=1.745184308 podStartE2EDuration="8.39914673s" totalImagesPullingTime="6.653962422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 18:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:26:32.398286817 +0000 UTC m=+4315.247793178" watchObservedRunningTime="2026-07-16 18:26:32.39914673 +0000 UTC m=+4315.248653139" Jul 16 18:26:41.440829 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:41.440787 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:41.716486 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:41.716410 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:41.803698 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:41.803672 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:42.842950 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:42.842924 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:43.105712 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:43.105638 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:43.215879 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:43.215850 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:44.284159 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:44.284123 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:44.487262 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:44.487228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:44.551683 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:44.551589 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:45.595897 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:45.595869 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:45.804587 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:45.804555 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:45.868101 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:45.867998 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:46.901825 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:46.901766 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:47.117769 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:47.117741 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:47.189681 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:47.189607 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:48.312694 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:48.312660 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:48.522509 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:48.522480 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:48.605450 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:48.605371 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:49.697132 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:49.697090 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:49.912747 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:49.912715 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:49.984930 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:49.984829 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:50.986144 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:50.986107 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:51.188180 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:51.188145 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:51.274457 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:51.274378 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:52.331113 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:52.331084 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:52.533756 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:52.533729 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:52.599216 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:52.599138 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:53.638465 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:53.638435 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:53.840407 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:53.840376 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:53.902184 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:53.902108 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:54.911568 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:54.911540 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:55.123364 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:55.123331 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:55.186882 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:55.186778 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:56.205751 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:56.205723 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:56.407695 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:56.407649 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:56.470718 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:56.470648 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:57.479758 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:57.479731 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:57.688272 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:57.688248 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:57.751686 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:57.751613 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:26:58.799957 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:58.799929 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:26:58.999402 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:58.999379 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:26:59.065602 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:26:59.065545 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:27:00.046227 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:00.046201 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_fc-smoke-test-kserve-7f4687897-r5tjx_1e491ead-ea2e-47bf-ada9-3cc83480e879/main/0.log" Jul 16 18:27:00.257770 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:00.257739 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs_96495a17-94db-46ed-8b38-53c7c97e598e/main/0.log" Jul 16 18:27:00.324120 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:00.324032 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-54c47cdfc7-kfbtn_1686a54a-6448-42ed-ac6e-9c9356d279d4/istio-proxy/0.log" Jul 16 18:27:01.737265 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:01.737230 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/0.log" Jul 16 18:27:01.768275 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:01.768213 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/1.log" Jul 16 18:27:01.781349 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:01.781324 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568468f9f4-9h48n_9790177d-d4dc-417f-aa42-f714a625d827/router/0.log" Jul 16 18:27:02.929423 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:02.929388 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/0.log" Jul 16 18:27:02.970944 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:02.970918 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/1.log" Jul 16 18:27:02.984940 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:02.984917 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568468f9f4-9h48n_9790177d-d4dc-417f-aa42-f714a625d827/router/0.log" Jul 16 18:27:04.147220 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:04.147164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/0.log" Jul 16 18:27:04.191940 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:04.191916 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/1.log" Jul 16 18:27:04.204248 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:04.204232 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568468f9f4-9h48n_9790177d-d4dc-417f-aa42-f714a625d827/router/0.log" Jul 16 18:27:05.344172 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:05.344141 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/0.log" Jul 16 18:27:05.389918 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:05.389889 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/1.log" Jul 16 18:27:05.403377 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:05.403353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568468f9f4-9h48n_9790177d-d4dc-417f-aa42-f714a625d827/router/0.log" Jul 16 18:27:06.318153 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:06.318127 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:06.339174 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:06.339146 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:07.396611 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:07.396583 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:07.414712 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:07.414691 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:08.455188 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:08.455162 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:08.470575 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:08.470550 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:09.485830 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:09.485784 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:09.501942 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:09.501920 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:10.477588 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:10.477556 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:10.493880 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:10.493851 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:11.457446 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:11.457418 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:11.476041 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:11.476015 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:12.454450 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:12.454418 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:12.470494 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:12.470467 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:13.478403 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:13.478374 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:13.496165 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:13.496141 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:14.534989 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:14.534965 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:14.550551 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:14.550529 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:15.536352 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:15.536322 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:15.555002 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:15.554950 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:16.582517 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:16.582484 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:16.602656 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:16.602630 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:17.616942 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:17.616915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:17.633411 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:17.633363 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:18.622437 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:18.622411 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:18.642193 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:18.642159 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:19.615004 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:19.614976 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:19.634646 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:19.634618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:20.655403 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:20.655377 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:20.690781 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:20.690753 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:21.690021 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:21.689997 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:21.704927 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:21.704894 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:22.690060 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:22.690032 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:22.705991 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:22.705964 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:23.719189 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:23.719165 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:23.735057 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:23.735034 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:24.750415 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:24.750386 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:24.766777 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:24.766757 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:25.726850 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:25.726815 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:25.745065 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:25.745029 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:26.748572 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:26.748536 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:26.770939 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:26.770910 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:27.761702 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:27.761677 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:27.777190 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:27.777143 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:28.794258 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:28.794230 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:28.809338 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:28.809314 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:29.823025 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:29.822996 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:29.839145 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:29.839113 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:30.881199 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:30.881173 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:30.896689 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:30.896656 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:31.964751 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:31.964723 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:31.980636 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:31.980591 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:32.959594 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:32.959558 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:32.976061 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:32.976040 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:33.972566 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:33.972541 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:33.989490 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:33.989464 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:35.033375 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:35.033345 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:35.052330 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:35.052300 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:36.051641 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:36.051612 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:36.070771 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:36.070745 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:37.040804 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:37.040757 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:37.061573 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:37.061553 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:38.167577 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:38.167550 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:38.192450 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:38.192421 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:39.257655 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:39.257625 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:39.281583 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:39.281560 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:40.333222 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:40.333199 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:40.347537 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:40.347515 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:41.365590 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:41.365567 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:41.381944 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:41.381895 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:42.406331 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:42.406299 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:42.425846 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:42.425820 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:43.422694 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:43.422665 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:43.441015 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:43.440991 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:44.442456 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:44.442432 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:44.457710 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:44.457682 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:45.447234 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:45.447203 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:45.468083 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:45.468055 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:46.456748 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:46.456723 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:46.471843 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:46.471808 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:47.500244 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:47.500202 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:47.522181 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:47.522159 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:48.558707 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:48.558677 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:48.574753 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:48.574728 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:49.570405 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:49.570380 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:49.585070 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:49.585047 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:50.587213 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:50.587185 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:50.604181 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:50.604158 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:51.562526 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:51.562495 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:51.582652 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:51.582624 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:52.555314 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:52.555288 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:52.572054 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:52.572031 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:53.632836 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:53.632784 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:53.652193 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:53.652142 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:54.683342 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:54.683310 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:54.698850 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:54.698828 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:55.722895 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:55.722869 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:55.739263 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:55.739236 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:56.806213 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:56.806185 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:56.822689 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:56.822661 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:57.852450 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:57.852421 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:57.869129 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:57.869103 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:58.908843 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:58.908807 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:58.930597 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:58.930562 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:27:59.929532 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:59.929494 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:27:59.946393 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:27:59.946367 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:00.919538 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:00.919506 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:00.937056 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:00.937023 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:01.979713 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:01.979687 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:01.996282 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:01.996257 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:03.021406 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:03.021376 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:03.037123 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:03.037096 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:04.058909 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:04.058884 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:04.077979 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:04.077952 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:05.108965 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:05.108936 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:05.125405 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:05.125379 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:06.175547 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:06.175514 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:06.207730 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:06.207704 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:07.276635 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:07.276602 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:07.308773 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:07.308746 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:08.327271 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:08.327244 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:08.342472 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:08.342444 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:09.886255 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:09.886224 2577 generic.go:356] "Generic (PLEG): container finished" podID="1077b595-1192-44e0-89fc-8044e905eda5" containerID="f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373" exitCode=0 Jul 16 18:28:09.886695 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:09.886297 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" event={"ID":"1077b595-1192-44e0-89fc-8044e905eda5","Type":"ContainerDied","Data":"f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373"} Jul 16 18:28:09.886695 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:09.886634 2577 scope.go:117] "RemoveContainer" containerID="f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373" Jul 16 18:28:09.942170 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:09.942145 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j9rsm_must-gather-w7m6q_1077b595-1192-44e0-89fc-8044e905eda5/gather/0.log" Jul 16 18:28:10.501705 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:10.501669 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:28:10.651392 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:10.651355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:28:10.752860 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:10.752755 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:28:14.710035 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:14.709998 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rp2zv_e9f16af0-c690-4e38-955a-2aae0c93c87c/global-pull-secret-syncer/0.log" Jul 16 18:28:14.756300 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:14.756264 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" Jul 16 18:28:14.947323 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:14.947291 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-160.ec2.internal_df2695e99c9c68ac5fa2b0530a286936/haproxy/0.log" Jul 16 18:28:15.458687 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.458647 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-j9rsm/must-gather-w7m6q"] Jul 16 18:28:15.458935 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.458907 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" podUID="1077b595-1192-44e0-89fc-8044e905eda5" containerName="copy" containerID="cri-o://7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9" gracePeriod=2 Jul 16 18:28:15.461283 ip-10-0-133-160 kubenswrapper[2577]: E0716 18:28:15.461241 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-w7m6q\" is forbidden: User \"system:node:ip-10-0-133-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j9rsm\": no relationship found between node 'ip-10-0-133-160.ec2.internal' and this object" podUID="1077b595-1192-44e0-89fc-8044e905eda5" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:28:15.461707 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.461687 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-j9rsm/must-gather-w7m6q"] Jul 16 18:28:15.690911 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.690885 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j9rsm_must-gather-w7m6q_1077b595-1192-44e0-89fc-8044e905eda5/copy/0.log" Jul 16 18:28:15.693343 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.693324 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:28:15.773808 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.773705 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1077b595-1192-44e0-89fc-8044e905eda5-must-gather-output\") pod \"1077b595-1192-44e0-89fc-8044e905eda5\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " Jul 16 18:28:15.774182 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.773811 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47c9v\" (UniqueName: \"kubernetes.io/projected/1077b595-1192-44e0-89fc-8044e905eda5-kube-api-access-47c9v\") pod \"1077b595-1192-44e0-89fc-8044e905eda5\" (UID: \"1077b595-1192-44e0-89fc-8044e905eda5\") " Jul 16 18:28:15.776299 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.776268 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1077b595-1192-44e0-89fc-8044e905eda5-kube-api-access-47c9v" (OuterVolumeSpecName: "kube-api-access-47c9v") pod "1077b595-1192-44e0-89fc-8044e905eda5" (UID: "1077b595-1192-44e0-89fc-8044e905eda5"). InnerVolumeSpecName "kube-api-access-47c9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 18:28:15.780951 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.780926 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1077b595-1192-44e0-89fc-8044e905eda5-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "1077b595-1192-44e0-89fc-8044e905eda5" (UID: "1077b595-1192-44e0-89fc-8044e905eda5"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 18:28:15.875607 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.875560 2577 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1077b595-1192-44e0-89fc-8044e905eda5-must-gather-output\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:28:15.875607 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.875585 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47c9v\" (UniqueName: \"kubernetes.io/projected/1077b595-1192-44e0-89fc-8044e905eda5-kube-api-access-47c9v\") on node \"ip-10-0-133-160.ec2.internal\" DevicePath \"\"" Jul 16 18:28:15.916864 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.916834 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j9rsm_must-gather-w7m6q_1077b595-1192-44e0-89fc-8044e905eda5/copy/0.log" Jul 16 18:28:15.917234 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.917210 2577 generic.go:356] "Generic (PLEG): container finished" podID="1077b595-1192-44e0-89fc-8044e905eda5" containerID="7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9" exitCode=143 Jul 16 18:28:15.917373 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.917280 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j9rsm/must-gather-w7m6q" Jul 16 18:28:15.917373 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.917328 2577 scope.go:117] "RemoveContainer" containerID="7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9" Jul 16 18:28:15.925866 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.925844 2577 scope.go:117] "RemoveContainer" containerID="f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373" Jul 16 18:28:15.942053 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.942029 2577 scope.go:117] "RemoveContainer" containerID="7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9" Jul 16 18:28:15.942425 ip-10-0-133-160 kubenswrapper[2577]: E0716 18:28:15.942391 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9\": container with ID starting with 7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9 not found: ID does not exist" containerID="7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9" Jul 16 18:28:15.942557 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.942432 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9"} err="failed to get container status \"7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9\": rpc error: code = NotFound desc = could not find container \"7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9\": container with ID starting with 7e36c66044e38f1a6b604512b99f62452701e8efc14352b52ace1d58ecf6afd9 not found: ID does not exist" Jul 16 18:28:15.942557 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.942456 2577 scope.go:117] "RemoveContainer" containerID="f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373" Jul 16 18:28:15.942743 ip-10-0-133-160 kubenswrapper[2577]: E0716 18:28:15.942725 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373\": container with ID starting with f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373 not found: ID does not exist" containerID="f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373" Jul 16 18:28:15.942835 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:15.942749 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373"} err="failed to get container status \"f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373\": rpc error: code = NotFound desc = could not find container \"f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373\": container with ID starting with f72d3214bf79f2872e859d4a90c725348491a1471e8fefb9fce669505472a373 not found: ID does not exist" Jul 16 18:28:17.369907 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:17.369873 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:28:17.546929 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:17.546902 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjqs9_fe344e5a-2ae4-454a-8be3-bf168ef3ed1d/konnectivity-agent/0.log" limit=1024 Jul 16 18:28:17.688273 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:17.688184 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1077b595-1192-44e0-89fc-8044e905eda5" path="/var/lib/kubelet/pods/1077b595-1192-44e0-89fc-8044e905eda5/volumes" Jul 16 18:28:20.556893 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:20.556862 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-69574b596d-nzkh5_6c9c1bde-6da0-4e69-b3db-b7cf39c0d3fb/limitador/0.log" Jul 16 18:28:20.599399 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:20.599372 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6f9f468797-d7wdw_14031588-b64f-4567-a3bd-c151b62aa493/manager/0.log" Jul 16 18:28:22.045335 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.045309 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-8cvcc_11dad2a0-1fb6-4fd7-a197-3ba2757fc659/kube-state-metrics/0.log" Jul 16 18:28:22.072267 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.072182 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-8cvcc_11dad2a0-1fb6-4fd7-a197-3ba2757fc659/kube-rbac-proxy-main/0.log" Jul 16 18:28:22.097660 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.097626 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-8cvcc_11dad2a0-1fb6-4fd7-a197-3ba2757fc659/kube-rbac-proxy-self/0.log" Jul 16 18:28:22.161233 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.161210 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-zpmld_7d8b5e0c-5719-4f22-98fe-7a64f3e53277/monitoring-plugin/0.log" Jul 16 18:28:22.304046 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.304020 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-chbml_f67e6150-2be9-40f6-825a-25944c25450d/node-exporter/0.log" Jul 16 18:28:22.329170 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.329108 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-chbml_f67e6150-2be9-40f6-825a-25944c25450d/kube-rbac-proxy/0.log" Jul 16 18:28:22.363123 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.363103 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-chbml_f67e6150-2be9-40f6-825a-25944c25450d/init-textfile/0.log" Jul 16 18:28:22.631833 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.631811 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8914c5b-dd02-4307-ab75-84b8f6103bfe/prometheus/0.log" Jul 16 18:28:22.655722 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.655703 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8914c5b-dd02-4307-ab75-84b8f6103bfe/config-reloader/0.log" Jul 16 18:28:22.679366 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.679349 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8914c5b-dd02-4307-ab75-84b8f6103bfe/thanos-sidecar/0.log" Jul 16 18:28:22.704400 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.704377 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8914c5b-dd02-4307-ab75-84b8f6103bfe/kube-rbac-proxy-web/0.log" Jul 16 18:28:22.734369 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.734355 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8914c5b-dd02-4307-ab75-84b8f6103bfe/kube-rbac-proxy/0.log" Jul 16 18:28:22.763798 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.763777 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8914c5b-dd02-4307-ab75-84b8f6103bfe/kube-rbac-proxy-thanos/0.log" Jul 16 18:28:22.793043 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.793027 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_c8914c5b-dd02-4307-ab75-84b8f6103bfe/init-config-reloader/0.log" Jul 16 18:28:22.825540 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.825511 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-z55bt_5801e7a4-b71f-4259-92e0-6f9fe9f159cb/prometheus-operator/0.log" Jul 16 18:28:22.853631 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.853555 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-z55bt_5801e7a4-b71f-4259-92e0-6f9fe9f159cb/kube-rbac-proxy/0.log" Jul 16 18:28:22.923087 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.923066 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b9c457dbb-5zf5d_eade205c-7424-467c-9ffe-3a16ff098a52/telemeter-client/0.log" Jul 16 18:28:22.958656 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.958639 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b9c457dbb-5zf5d_eade205c-7424-467c-9ffe-3a16ff098a52/reload/0.log" Jul 16 18:28:22.985460 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:22.985439 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b9c457dbb-5zf5d_eade205c-7424-467c-9ffe-3a16ff098a52/kube-rbac-proxy/0.log" Jul 16 18:28:24.577598 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.577564 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk"] Jul 16 18:28:24.577994 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.577916 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1077b595-1192-44e0-89fc-8044e905eda5" containerName="gather" Jul 16 18:28:24.577994 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.577928 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1077b595-1192-44e0-89fc-8044e905eda5" containerName="gather" Jul 16 18:28:24.577994 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.577953 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1077b595-1192-44e0-89fc-8044e905eda5" containerName="copy" Jul 16 18:28:24.577994 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.577959 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1077b595-1192-44e0-89fc-8044e905eda5" containerName="copy" Jul 16 18:28:24.578129 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.578007 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1077b595-1192-44e0-89fc-8044e905eda5" containerName="copy" Jul 16 18:28:24.578129 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.578018 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1077b595-1192-44e0-89fc-8044e905eda5" containerName="gather" Jul 16 18:28:24.584726 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.584702 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.587402 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.587379 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qh77t\"/\"default-dockercfg-m6ppb\"" Jul 16 18:28:24.588463 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.588439 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh77t\"/\"openshift-service-ca.crt\"" Jul 16 18:28:24.588814 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.588466 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qh77t\"/\"kube-root-ca.crt\"" Jul 16 18:28:24.593872 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.590365 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk"] Jul 16 18:28:24.655421 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.655394 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-lib-modules\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.655533 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.655468 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-sys\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.655533 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.655508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-proc\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.655533 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.655530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dhq7\" (UniqueName: \"kubernetes.io/projected/9fa2d912-bdee-4f54-9ad6-2b541329339a-kube-api-access-2dhq7\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.655658 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.655605 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-podres\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756412 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-sys\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756527 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-proc\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756527 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dhq7\" (UniqueName: \"kubernetes.io/projected/9fa2d912-bdee-4f54-9ad6-2b541329339a-kube-api-access-2dhq7\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756527 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756499 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-sys\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756674 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756521 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-proc\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756674 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-podres\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756674 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-lib-modules\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756850 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756720 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-podres\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.756850 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.756748 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fa2d912-bdee-4f54-9ad6-2b541329339a-lib-modules\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.765405 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.765383 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dhq7\" (UniqueName: \"kubernetes.io/projected/9fa2d912-bdee-4f54-9ad6-2b541329339a-kube-api-access-2dhq7\") pod \"perf-node-gather-daemonset-9vqbk\" (UID: \"9fa2d912-bdee-4f54-9ad6-2b541329339a\") " pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:24.900659 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:24.900638 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:25.029808 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:25.029687 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk"] Jul 16 18:28:25.032101 ip-10-0-133-160 kubenswrapper[2577]: W0716 18:28:25.032073 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9fa2d912_bdee_4f54_9ad6_2b541329339a.slice/crio-c57a82f3c2446cc96468ec75b8d164bd25ed56ec81c1eaa2b720327d7f780649 WatchSource:0}: Error finding container c57a82f3c2446cc96468ec75b8d164bd25ed56ec81c1eaa2b720327d7f780649: Status 404 returned error can't find the container with id c57a82f3c2446cc96468ec75b8d164bd25ed56ec81c1eaa2b720327d7f780649 Jul 16 18:28:25.963724 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:25.963687 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" event={"ID":"9fa2d912-bdee-4f54-9ad6-2b541329339a","Type":"ContainerStarted","Data":"9b3f05094e77c08ca5c649eedf08728bc898e9578862ebe51e064e9af0a4d201"} Jul 16 18:28:25.964093 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:25.963750 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:25.964093 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:25.963771 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" event={"ID":"9fa2d912-bdee-4f54-9ad6-2b541329339a","Type":"ContainerStarted","Data":"c57a82f3c2446cc96468ec75b8d164bd25ed56ec81c1eaa2b720327d7f780649"} Jul 16 18:28:25.981583 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:25.981534 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" podStartSLOduration=1.981520568 podStartE2EDuration="1.981520568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 18:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 18:28:25.980750926 +0000 UTC m=+4428.830257299" watchObservedRunningTime="2026-07-16 18:28:25.981520568 +0000 UTC m=+4428.831026927" Jul 16 18:28:27.223691 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:27.223660 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4h2x5_eb84a249-ddf6-430e-bf26-2496fdf6b5bd/dns/0.log" Jul 16 18:28:27.251863 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:27.251836 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4h2x5_eb84a249-ddf6-430e-bf26-2496fdf6b5bd/kube-rbac-proxy/0.log" Jul 16 18:28:27.388081 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:27.388059 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7w5mz_5a8f2521-2a93-4462-89b6-e2d8aced60d5/dns-node-resolver/0.log" Jul 16 18:28:28.111786 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:28.111760 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rv7rf_0bd348bf-425d-45d0-8fe0-a91dfc261aae/node-ca/0.log" Jul 16 18:28:29.661591 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:29.661545 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/0.log" Jul 16 18:28:29.693385 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:29.693352 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/1.log" Jul 16 18:28:29.719450 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:29.719427 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568468f9f4-9h48n_9790177d-d4dc-417f-aa42-f714a625d827/router/0.log" Jul 16 18:28:30.326961 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:30.326910 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fc7ll_5a5c642e-79dd-419a-b4d0-c309f7988d04/serve-healthcheck-canary/0.log" Jul 16 18:28:30.890931 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:30.890904 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-dwrfw_53f52dc0-37b5-4f63-814c-599b8fd74feb/insights-operator/0.log" Jul 16 18:28:30.897820 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:30.893173 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-dwrfw_53f52dc0-37b5-4f63-814c-599b8fd74feb/insights-operator/1.log" Jul 16 18:28:31.116257 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:31.116221 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9qkd_f32c35b6-d851-4c34-b4b6-5586ca0e2357/kube-rbac-proxy/0.log" Jul 16 18:28:31.150286 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:31.150211 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9qkd_f32c35b6-d851-4c34-b4b6-5586ca0e2357/exporter/0.log" Jul 16 18:28:31.182520 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:31.182499 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9qkd_f32c35b6-d851-4c34-b4b6-5586ca0e2357/extractor/0.log" Jul 16 18:28:31.978341 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:31.978312 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qh77t/perf-node-gather-daemonset-9vqbk" Jul 16 18:28:34.647733 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:34.647700 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/0.log" Jul 16 18:28:34.676954 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:34.676929 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/1.log" Jul 16 18:28:34.689657 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:34.689633 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568468f9f4-9h48n_9790177d-d4dc-417f-aa42-f714a625d827/router/0.log" Jul 16 18:28:34.714400 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:34.714370 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-558b5fdc8c-pv4cf_5dd0572f-2478-4da0-b0ec-56cb53404123/manager/0.log" Jul 16 18:28:36.299775 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:36.299744 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/0.log" Jul 16 18:28:36.328756 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:36.328731 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-769cdd8d88-mfr9j_3fcc4e1a-8fd5-40c1-98dc-7119ecc39f20/istio-proxy/1.log" Jul 16 18:28:36.354076 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:36.354045 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-568468f9f4-9h48n_9790177d-d4dc-417f-aa42-f714a625d827/router/0.log" Jul 16 18:28:41.220745 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:41.220710 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-klqmp_efae4550-d04d-49c5-9cee-f54b06131155/migrator/0.log" Jul 16 18:28:41.247949 ip-10-0-133-160 kubenswrapper[2577]: I0716 18:28:41.247926 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-klqmp_efae4550-d04d-49c5-9cee-f54b06131155/graceful-termination/0.log"